Jul 15 13:51:01.456571 ip-10-0-133-21 systemd[1]: Starting Kubernetes Kubelet... Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.821170 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825443 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825454 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825459 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825462 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825464 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825468 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825471 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 13:51:01.940195 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825473 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 13:51:01.892406 ip-10-0-133-21 systemd[1]: Started Kubernetes Kubelet. Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825476 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825478 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825481 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825483 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825486 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825488 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825491 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825493 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825496 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825499 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825502 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825505 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825508 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825511 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825513 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825516 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825519 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825522 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825525 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 13:51:01.941071 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825528 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825530 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825533 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825535 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825538 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825541 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825544 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825547 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825550 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825552 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825555 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825558 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825560 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825563 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825566 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825568 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825571 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825574 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825577 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825579 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 13:51:01.941813 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825581 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825584 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825587 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825591 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825593 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825595 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825598 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825601 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825605 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825608 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825611 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825614 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825617 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825620 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825622 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825625 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825630 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825634 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825637 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 13:51:01.942452 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825640 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825642 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825645 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825648 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825650 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825653 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825656 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825659 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825661 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825664 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825666 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825669 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825671 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825674 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825676 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825679 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825682 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825684 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825687 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825690 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 13:51:01.943082 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825693 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825695 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825698 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825701 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825703 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825706 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825708 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825711 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825713 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825716 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825719 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825722 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.825726 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826439 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826445 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826450 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826455 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826458 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826462 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 13:51:01.943677 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826465 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826469 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826471 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826474 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826476 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826479 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826482 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826484 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826488 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826490 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826493 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826495 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826498 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826501 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826504 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826509 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826512 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826515 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826518 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 13:51:01.944254 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826521 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826524 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826526 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826530 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826533 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826535 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826538 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826541 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826543 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826547 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826549 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826553 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826555 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826560 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826566 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826569 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826572 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826575 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826578 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826580 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 13:51:01.944819 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826583 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826586 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826589 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826592 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826594 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826597 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826600 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826602 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826605 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826608 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826611 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826614 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826617 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826619 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826622 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826624 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826627 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826630 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826633 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826635 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 13:51:01.945523 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826638 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826640 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826643 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826646 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826649 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826651 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826655 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826658 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826661 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826663 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826666 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826668 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826671 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826673 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826676 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826678 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826681 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826683 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826686 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 13:51:01.946248 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826688 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826691 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826693 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826696 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826700 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826703 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826705 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826708 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826710 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826713 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826715 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826718 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826720 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826723 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.826725 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827142 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827149 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827153 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827157 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 13:51:01.946928 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827162 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827165 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827168 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827171 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827174 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827177 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827180 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827182 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827185 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827188 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827192 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827194 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827197 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827200 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827203 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827205 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827207 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827211 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827214 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 13:51:01.947766 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827217 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827220 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827222 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827225 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827227 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827229 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827233 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827235 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827238 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827240 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827243 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827245 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827248 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827252 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827255 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827258 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827260 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827263 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827265 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827268 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 13:51:01.948465 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827270 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827273 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827276 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827278 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827281 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827283 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827285 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827288 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827290 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827307 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827311 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827314 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827316 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827319 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827322 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827325 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827327 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827330 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827332 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827335 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 13:51:01.949256 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827337 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827340 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827342 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827345 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827348 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827352 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827355 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827358 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827360 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827363 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827365 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827368 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827373 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827377 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827380 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827383 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827386 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827389 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827392 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 13:51:01.949849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827394 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827397 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827399 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827403 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827406 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827410 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827413 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827416 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827419 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827422 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827425 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827427 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827430 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827432 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827435 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827437 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.827440 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827499 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827507 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827514 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827519 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 13:51:02.134394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827523 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827526 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827531 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827535 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827538 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827540 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827544 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827547 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827550 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827553 2584 flags.go:64] FLAG: --cgroup-root="" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827556 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827559 2584 flags.go:64] FLAG: --client-ca-file="" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827562 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827565 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827571 2584 flags.go:64] FLAG: --cluster-domain="" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827574 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827577 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827580 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827583 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827587 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827590 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827593 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827596 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827600 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 15 13:51:02.158030 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827603 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827606 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827609 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827612 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827615 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827620 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827623 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827626 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827629 2584 flags.go:64] FLAG: --enable-server="true" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827632 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827636 2584 flags.go:64] FLAG: --event-burst="100" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827639 2584 flags.go:64] FLAG: --event-qps="50" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827642 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827646 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827648 2584 flags.go:64] FLAG: --eviction-hard="" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827652 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827655 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827658 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827661 2584 flags.go:64] FLAG: --eviction-soft="" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827664 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827666 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827669 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827674 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827677 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827680 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 13:51:02.158853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827682 2584 flags.go:64] FLAG: --feature-gates="" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827686 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827689 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827692 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827695 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827698 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827701 2584 flags.go:64] FLAG: --help="false" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827704 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827707 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827710 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827713 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827716 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827721 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827723 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827726 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827729 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827731 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827735 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827738 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827741 2584 flags.go:64] FLAG: --kube-reserved="" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827744 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827747 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827749 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827752 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 13:51:02.159839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827755 2584 flags.go:64] FLAG: --lock-file="" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827758 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827760 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827763 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827768 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827773 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827776 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827779 2584 flags.go:64] FLAG: --logging-format="text" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827781 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827785 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827788 2584 flags.go:64] FLAG: --manifest-url="" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827790 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827794 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827797 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827801 2584 flags.go:64] FLAG: --max-pods="110" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827804 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827807 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827810 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827813 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827815 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827821 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827824 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827831 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827834 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827837 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 13:51:02.160548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827840 2584 flags.go:64] FLAG: --pod-cidr="" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827843 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827849 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827852 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827855 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827858 2584 flags.go:64] FLAG: --port="10250" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827861 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827863 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05e0d023404c9a5ed" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827866 2584 flags.go:64] FLAG: --qos-reserved="" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827869 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827872 2584 flags.go:64] FLAG: --register-node="true" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827875 2584 flags.go:64] FLAG: --register-with-taints="" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827880 2584 flags.go:64] FLAG: --registry-burst="10" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827883 2584 flags.go:64] FLAG: --registry-qps="5" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827886 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827889 2584 flags.go:64] FLAG: --reserved-memory="" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827893 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827896 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827899 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827902 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827904 2584 flags.go:64] FLAG: --runonce="false" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827907 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827910 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827913 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827916 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 13:51:02.161257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827919 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827922 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827925 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827929 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827932 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827935 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827938 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827941 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827943 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827947 2584 flags.go:64] FLAG: --system-cgroups="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827950 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827955 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827958 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827960 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827964 2584 flags.go:64] FLAG: --tls-min-version="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827967 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827970 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827973 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827975 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827980 2584 flags.go:64] FLAG: --v="2" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827983 2584 flags.go:64] FLAG: --version="false" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827987 2584 flags.go:64] FLAG: --vmodule="" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827991 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.827994 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828108 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 13:51:02.161937 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828112 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828116 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828120 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828123 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828126 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828129 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828131 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828134 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828136 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828139 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828143 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828145 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828148 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828150 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828153 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828155 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828158 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828161 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828164 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 13:51:02.162644 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828166 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828169 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828172 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828174 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828177 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828179 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828182 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828186 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828188 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828191 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828195 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828198 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828201 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828204 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828207 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828210 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828213 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828215 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828218 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 13:51:02.163156 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828221 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828223 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828226 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828230 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828235 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828239 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828242 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828245 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828247 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828250 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828253 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828255 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828258 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828261 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828264 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828266 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828269 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828271 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828274 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 13:51:02.163725 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828276 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828280 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828282 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828285 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828287 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828290 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828306 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828309 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828312 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828314 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828317 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828319 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828322 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828325 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828327 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828330 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828333 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828339 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828341 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828344 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 13:51:02.164226 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828346 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828349 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828351 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828354 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828357 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828359 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828362 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828365 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828368 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828370 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828373 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828375 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828377 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828381 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828384 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828386 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828389 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828392 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828394 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828397 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 13:51:02.164811 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.828399 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.828405 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.835785 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.835801 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.835821 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.835831 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835891 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835897 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835901 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835904 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835908 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835911 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835914 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835916 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 13:51:02.165418 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835919 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835921 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835924 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835927 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835929 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835932 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835935 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835937 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835940 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835943 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835945 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835950 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835953 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835956 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835958 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835960 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835963 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835966 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835968 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835970 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 13:51:02.165858 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835973 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835982 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835985 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835987 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835990 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835993 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835995 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.835998 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836002 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836005 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836008 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836012 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836015 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836017 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836020 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836023 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836025 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836028 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836030 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836033 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 13:51:02.166437 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836035 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836038 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836041 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836047 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836050 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836052 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836055 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836058 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836060 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836063 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836065 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836068 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836070 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836072 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836080 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836083 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836086 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836088 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836091 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 13:51:02.167022 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836093 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836097 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836100 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836102 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836105 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836107 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836110 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836112 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836115 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836117 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836120 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836122 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836124 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836127 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836129 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836132 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836136 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836138 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836141 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836143 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 13:51:02.167537 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836146 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836148 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836151 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836153 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836156 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836158 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836161 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836163 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836171 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836174 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836177 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836179 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.836184 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836345 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836362 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 13:51:02.168105 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836365 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836368 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836370 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836373 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836376 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836378 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836381 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836383 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836386 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836388 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836391 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836393 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836396 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836399 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836402 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836405 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836407 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836410 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836412 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836415 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 13:51:02.168577 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836417 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836420 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836422 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836424 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836427 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836429 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836438 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836441 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836443 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836446 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836448 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836452 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836455 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836457 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836460 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836462 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836465 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836468 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836470 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836473 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 13:51:02.169140 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836475 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836477 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836480 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836482 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836486 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836490 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836494 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836497 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836500 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836502 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836505 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836507 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836510 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836512 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836515 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836517 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836521 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836524 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836527 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 13:51:02.170054 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836537 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836540 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836543 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836546 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836549 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836552 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836555 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836558 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836560 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836563 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836566 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836568 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836571 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836573 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836576 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836580 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836583 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836585 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836588 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836592 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 13:51:02.424540 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836594 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836597 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836600 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836602 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836605 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836607 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836610 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836613 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836615 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836617 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836620 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836623 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836625 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836633 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836635 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836638 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836642 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 13:51:02.426390 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:01.836644 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.836649 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.836779 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.840690 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.841978 2584 server.go:1019] "Starting client certificate rotation" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.842077 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.842112 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.845779 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.847052 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.857167 2584 log.go:25] "Validated CRI v1 runtime API" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.861913 2584 log.go:25] "Validated CRI v1 image API" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.862623 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 13:51:02.426928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.864830 2584 fs.go:135] Filesystem UUIDs: map[4801a64e-6123-4748-a427-ae8f4c6ab757:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d384cca4-5fc7-4fb4-b4c4-7f04a7dbffd4:/dev/nvme0n1p4] Jul 15 13:51:02.427309 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.864844 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.870124 2584 manager.go:216] Machine: {Timestamp:2026-07-15 13:51:01.868869726 +0000 UTC m=+0.306478509 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100169 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec237b036266020c39093ecbe4d0fe4f SystemUUID:ec237b03-6266-020c-3909-3ecbe4d0fe4f BootID:f3f8d208-4223-4c94-a51b-3b5c9449248a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f7:a1:ad:c4:d9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f7:a1:ad:c4:d9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:a4:91:35:e6:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.870241 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.870333 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.872429 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.872454 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-21.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.876873 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.876886 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 13:51:02.427440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.876913 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.876917 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.877550 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.878408 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882545 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882650 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882678 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882686 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882700 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882709 2584 kubelet.go:398] "Adding apiserver pod source" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.882717 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.883000 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.883013 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.884838 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887051 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887080 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887743 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887757 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887763 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887768 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887776 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887781 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887789 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887794 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887800 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887807 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887816 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.887823 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.888512 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.888518 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.891542 2584 server.go:1268] "Started kubelet" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.891629 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.891700 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.891785 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.895408 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.895990 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.896821 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-21.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.896831 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.896919 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 13:51:02.427747 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.896977 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.896805 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-21.ec2.internal.18c27ad5fd7beba9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-21.ec2.internal,UID:ip-10-0-133-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-21.ec2.internal,},FirstTimestamp:2026-07-15 13:51:01.891517353 +0000 UTC m=+0.329126136,LastTimestamp:2026-07-15 13:51:01.891517353 +0000 UTC m=+0.329126136,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-21.ec2.internal,}" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.899588 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.899597 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.899643 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900194 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900210 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900211 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900327 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900336 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900636 2584 factory.go:153] Registering CRI-O factory Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900651 2584 factory.go:223] Registration of the crio container factory successfully Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900699 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900709 2584 factory.go:55] Registering systemd factory Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900717 2584 factory.go:223] Registration of the systemd container factory successfully Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900740 2584 factory.go:103] Registering Raw factory Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.900753 2584 manager.go:1196] Started watching for new ooms in manager Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.901193 2584 manager.go:349] Starting recovery of all containers Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.910964 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7k2x" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.912819 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.917216 2584 manager.go:354] Recovery completed Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.918651 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.918654 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7k2x" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.918821 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.924151 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926283 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926331 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926343 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926733 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926743 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926758 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.926770 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.927523 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.429032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.927549 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.927562 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.928841 2584 policy_none.go:49] "None policy: Start" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.928853 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.928877 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.930899 2584 policy_none.go:47] "Start" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.969959 2584 manager.go:337] "Starting Device Plugin manager" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.970226 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.970238 2584 server.go:87] "Starting device plugin registration server" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.970591 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.970607 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.971091 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.971181 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:01.971191 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.971836 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:01.971885 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.001096 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.001134 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.001152 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.001181 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.003835 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.070918 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.072355 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.072382 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.072393 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.072425 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.090780 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.090798 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-21.ec2.internal\": node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.101578 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal"] Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.101626 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.102396 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.102417 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.102429 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.104591 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.104704 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.104726 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.105223 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.105245 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.430081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.105257 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.106068 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.106091 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.106106 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.108250 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.108278 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.109530 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientMemory" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.109558 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.109571 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeHasSufficientPID" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.119267 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.124849 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-21.ec2.internal\" not found" node="ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.130686 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-21.ec2.internal\" not found" node="ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.201976 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/81cd5de19bf3e7dbbe6adfa180df0e0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal\" (UID: \"81cd5de19bf3e7dbbe6adfa180df0e0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.202002 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81cd5de19bf3e7dbbe6adfa180df0e0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal\" (UID: \"81cd5de19bf3e7dbbe6adfa180df0e0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.202027 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72a36e4b47671bef90a84d11574ce86d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-21.ec2.internal\" (UID: \"72a36e4b47671bef90a84d11574ce86d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.220071 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.302439 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72a36e4b47671bef90a84d11574ce86d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-21.ec2.internal\" (UID: \"72a36e4b47671bef90a84d11574ce86d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.302465 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/81cd5de19bf3e7dbbe6adfa180df0e0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal\" (UID: \"81cd5de19bf3e7dbbe6adfa180df0e0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.302490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81cd5de19bf3e7dbbe6adfa180df0e0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal\" (UID: \"81cd5de19bf3e7dbbe6adfa180df0e0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.302533 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81cd5de19bf3e7dbbe6adfa180df0e0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal\" (UID: \"81cd5de19bf3e7dbbe6adfa180df0e0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431288 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.302563 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72a36e4b47671bef90a84d11574ce86d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-21.ec2.internal\" (UID: \"72a36e4b47671bef90a84d11574ce86d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431927 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.302570 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/81cd5de19bf3e7dbbe6adfa180df0e0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal\" (UID: \"81cd5de19bf3e7dbbe6adfa180df0e0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.431927 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.320778 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.431927 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.421657 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.431927 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.428287 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.633511 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.433882 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:02.633511 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.523293 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.633511 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.623377 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.857387 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.724469 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.857387 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.825345 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-21.ec2.internal\" not found" Jul 15 13:51:02.857387 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.841717 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 13:51:02.857387 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.841876 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.870041 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.883377 2584 apiserver.go:52] "Watching apiserver" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.893404 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.894344 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-p4t6d","openshift-cluster-node-tuning-operator/tuned-pqv2j","openshift-dns/node-resolver-9whzn","openshift-multus/multus-k4k64","openshift-multus/network-metrics-daemon-ncnc9","kube-system/global-pull-secret-syncer-t7tnx","kube-system/konnectivity-agent-858gf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv","openshift-image-registry/node-ca-fl65z","openshift-multus/multus-additional-cni-plugins-26sc2","openshift-network-diagnostics/network-check-target-5qvnw","openshift-network-operator/iptables-alerter-9nbnd"] Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.897630 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.899932 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.899946 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.900648 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.901313 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.901378 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.901417 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.901487 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2sn6p\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.902341 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.902393 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.902600 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.903138 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.904925 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.905551 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2k4vz\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.905743 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907056 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-cni-netd\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907084 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907101 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-modprobe-d\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.098064 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907117 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysconfig\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907132 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-lib-modules\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907177 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-run-netns\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907203 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-systemd\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907223 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907258 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-env-overrides\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907275 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysctl-conf\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907290 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-systemd\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907320 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-run\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907337 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-host\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907392 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-etc-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907557 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-node-log\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907579 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjj4d\" (UniqueName: \"kubernetes.io/projected/039fd736-4e61-418b-9391-1d10f7401fc0-kube-api-access-kjj4d\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907596 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9618dd0e-33ad-4458-88d6-7442904afcf4-tmp-dir\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.100066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907622 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-kubelet\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-ovn\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907656 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-log-socket\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907670 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-ovnkube-script-lib\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907691 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysctl-d\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-sys\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907725 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-cni-bin\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907740 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/039fd736-4e61-418b-9391-1d10f7401fc0-ovn-node-metrics-cert\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907756 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-var-lib-kubelet\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.907990 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9618dd0e-33ad-4458-88d6-7442904afcf4-hosts-file\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908011 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-slash\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908019 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6wtwg\"" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908029 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-tuned\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908044 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba93ea4-b1a2-4de0-90f0-8065fb23c014-tmp\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.100742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28sqk\" (UniqueName: \"kubernetes.io/projected/dba93ea4-b1a2-4de0-90f0-8065fb23c014-kube-api-access-28sqk\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908107 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svwm7\" (UniqueName: \"kubernetes.io/projected/9618dd0e-33ad-4458-88d6-7442904afcf4-kube-api-access-svwm7\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908125 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-ovnkube-config\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908140 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-kubernetes\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908155 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-systemd-units\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908170 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-var-lib-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908186 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908339 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.908423 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.909439 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.911840 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.911930 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.911974 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.912050 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.914375 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.914433 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.915778 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.915846 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.915854 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x6xmf\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.915908 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.915965 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.101293 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.916807 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.919877 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.920791 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.921141 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.921239 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q6szc\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.922489 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.922492 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.922966 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.922984 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7rqvj\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.923079 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.923253 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.924601 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 13:46:02 +0000 UTC" deadline="2028-04-26 05:33:25.472134074 +0000 UTC" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.924623 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15615h42m22.547514945s" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.925813 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.926311 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.926449 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-948x6\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.927060 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.927374 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.929996 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.930232 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.930249 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r68pz\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.931096 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:02.931144 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.933787 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal"] Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.933877 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.934773 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal"] Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.934879 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.936653 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.936843 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:03.102514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.936991 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2h4ts\"" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.937245 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.980688 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c6nfk" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:02.989290 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c6nfk" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.001592 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008354 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008389 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-host\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008409 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-k8s-cni-cncf-io\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008427 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjj4d\" (UniqueName: \"kubernetes.io/projected/039fd736-4e61-418b-9391-1d10f7401fc0-kube-api-access-kjj4d\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008491 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-host\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008531 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skmjb\" (UniqueName: \"kubernetes.io/projected/4b85a86b-ed33-479c-a7f8-f07197ea822d-kube-api-access-skmjb\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008556 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf75f52c-b77a-4a9a-8b02-7e823c68380a-kubelet-config\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008592 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-kubelet\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008615 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5707ea40-64ff-4bf3-b355-32b3d29f80f8-konnectivity-ca\") pod \"konnectivity-agent-858gf\" (UID: \"5707ea40-64ff-4bf3-b355-32b3d29f80f8\") " pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008629 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-os-release\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008659 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-kubelet\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.103458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008661 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-host\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008684 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-log-socket\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008706 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-sys\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008739 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-serviceca\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008730 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-log-socket\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008758 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-sys\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008815 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-cni-bin\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-cni-bin\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008866 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/039fd736-4e61-418b-9391-1d10f7401fc0-ovn-node-metrics-cert\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008901 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngzdf\" (UniqueName: \"kubernetes.io/projected/01aac545-69a2-4eda-97dc-e2a200140a51-kube-api-access-ngzdf\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008920 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-socket-dir-parent\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008935 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-netns\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.104068 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008951 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-slash\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008983 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-tuned\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.008991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-slash\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009008 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba93ea4-b1a2-4de0-90f0-8065fb23c014-tmp\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009027 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svwm7\" (UniqueName: \"kubernetes.io/projected/9618dd0e-33ad-4458-88d6-7442904afcf4-kube-api-access-svwm7\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009052 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009071 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j6m2\" (UniqueName: \"kubernetes.io/projected/f8be98fb-fca8-4048-b97e-b042af16da82-kube-api-access-2j6m2\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009088 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtw2j\" (UniqueName: \"kubernetes.io/projected/b731e8a7-a469-4d64-9ac5-14ced255483d-kube-api-access-qtw2j\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009112 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-cni-netd\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009145 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009183 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009184 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-cni-netd\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009252 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysconfig\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.104651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009281 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-socket-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009326 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-system-cni-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009338 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysconfig\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009357 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-cni-multus\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009387 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009415 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009445 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-run-netns\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009466 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-systemd\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009511 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009546 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-env-overrides\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009559 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009568 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-systemd\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009610 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-run-netns\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009569 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysctl-conf\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-systemd\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009673 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-run\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009682 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysctl-conf\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009694 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-etc-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009717 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-node-log\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009727 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-systemd\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009739 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9618dd0e-33ad-4458-88d6-7442904afcf4-tmp-dir\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009763 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-etc-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009804 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-node-log\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009763 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-registration-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-run\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009841 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-system-cni-dir\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.009896 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-cnibin\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.010051 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-env-overrides\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.010062 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9618dd0e-33ad-4458-88d6-7442904afcf4-tmp-dir\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.105767 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.012733 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/039fd736-4e61-418b-9391-1d10f7401fc0-ovn-node-metrics-cert\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.014785 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba93ea4-b1a2-4de0-90f0-8065fb23c014-tmp\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.014832 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-tuned\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015444 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-os-release\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015478 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmhr6\" (UniqueName: \"kubernetes.io/projected/8fdeba2b-e439-4ece-82fb-d7f845764793-kube-api-access-mmhr6\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015494 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-cni-bin\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015511 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-conf-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015525 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf75f52c-b77a-4a9a-8b02-7e823c68380a-dbus\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015544 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-ovn\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015584 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-ovnkube-script-lib\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015602 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-run-ovn\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015621 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysctl-d\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015645 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015663 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-cni-binary-copy\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.106336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015709 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fdeba2b-e439-4ece-82fb-d7f845764793-host-slash\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015779 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-cni-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015781 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-sysctl-d\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015801 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-var-lib-kubelet\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015825 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9618dd0e-33ad-4458-88d6-7442904afcf4-hosts-file\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015839 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-var-lib-kubelet\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015841 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8be98fb-fca8-4048-b97e-b042af16da82-cni-binary-copy\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015865 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-kubelet\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015874 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9618dd0e-33ad-4458-88d6-7442904afcf4-hosts-file\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015886 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-hostroot\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015903 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-multus-certs\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-etc-kubernetes\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015933 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28sqk\" (UniqueName: \"kubernetes.io/projected/dba93ea4-b1a2-4de0-90f0-8065fb23c014-kube-api-access-28sqk\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.106821 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015949 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-device-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015968 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-sys-fs\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.015984 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-ovnkube-config\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016000 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-kubernetes\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016015 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8fdeba2b-e439-4ece-82fb-d7f845764793-iptables-alerter-script\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-cnibin\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016062 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f8be98fb-fca8-4048-b97e-b042af16da82-multus-daemon-config\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016084 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8hn4\" (UniqueName: \"kubernetes.io/projected/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-kube-api-access-n8hn4\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016082 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-ovnkube-script-lib\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016099 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-systemd-units\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016118 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-var-lib-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016136 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016166 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-modprobe-d\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016230 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-modprobe-d\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107331 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016255 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016289 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-etc-kubernetes\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016319 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-var-lib-openvswitch\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016349 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/039fd736-4e61-418b-9391-1d10f7401fc0-systemd-units\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016390 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-lib-modules\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016416 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5707ea40-64ff-4bf3-b355-32b3d29f80f8-agent-certs\") pod \"konnectivity-agent-858gf\" (UID: \"5707ea40-64ff-4bf3-b355-32b3d29f80f8\") " pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016501 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dba93ea4-b1a2-4de0-90f0-8065fb23c014-lib-modules\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.016511 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/039fd736-4e61-418b-9391-1d10f7401fc0-ovnkube-config\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.021933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjj4d\" (UniqueName: \"kubernetes.io/projected/039fd736-4e61-418b-9391-1d10f7401fc0-kube-api-access-kjj4d\") pod \"ovnkube-node-p4t6d\" (UID: \"039fd736-4e61-418b-9391-1d10f7401fc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.022013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svwm7\" (UniqueName: \"kubernetes.io/projected/9618dd0e-33ad-4458-88d6-7442904afcf4-kube-api-access-svwm7\") pod \"node-resolver-9whzn\" (UID: \"9618dd0e-33ad-4458-88d6-7442904afcf4\") " pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.033735 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28sqk\" (UniqueName: \"kubernetes.io/projected/dba93ea4-b1a2-4de0-90f0-8065fb23c014-kube-api-access-28sqk\") pod \"tuned-pqv2j\" (UID: \"dba93ea4-b1a2-4de0-90f0-8065fb23c014\") " pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.107930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.095004 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.116982 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117014 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-k8s-cni-cncf-io\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skmjb\" (UniqueName: \"kubernetes.io/projected/4b85a86b-ed33-479c-a7f8-f07197ea822d-kube-api-access-skmjb\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117059 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf75f52c-b77a-4a9a-8b02-7e823c68380a-kubelet-config\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5707ea40-64ff-4bf3-b355-32b3d29f80f8-konnectivity-ca\") pod \"konnectivity-agent-858gf\" (UID: \"5707ea40-64ff-4bf3-b355-32b3d29f80f8\") " pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117158 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf75f52c-b77a-4a9a-8b02-7e823c68380a-kubelet-config\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117146 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-os-release\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117111 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-k8s-cni-cncf-io\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117213 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-host\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117264 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-os-release\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117258 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-serviceca\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117333 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-host\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.311742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117358 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117409 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngzdf\" (UniqueName: \"kubernetes.io/projected/01aac545-69a2-4eda-97dc-e2a200140a51-kube-api-access-ngzdf\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117445 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117446 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-socket-dir-parent\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117485 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-netns\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117501 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-socket-dir-parent\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j6m2\" (UniqueName: \"kubernetes.io/projected/f8be98fb-fca8-4048-b97e-b042af16da82-kube-api-access-2j6m2\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117581 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtw2j\" (UniqueName: \"kubernetes.io/projected/b731e8a7-a469-4d64-9ac5-14ced255483d-kube-api-access-qtw2j\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.117598 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117616 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-socket-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117646 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-system-cni-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117650 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-netns\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117696 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-cni-multus\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117649 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.312316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117725 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-serviceca\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117783 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117804 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-socket-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117814 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117823 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5707ea40-64ff-4bf3-b355-32b3d29f80f8-konnectivity-ca\") pod \"konnectivity-agent-858gf\" (UID: \"5707ea40-64ff-4bf3-b355-32b3d29f80f8\") " pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117859 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-registration-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117888 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-cni-multus\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.117891 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.117907 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:03.617883769 +0000 UTC m=+2.055492556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117922 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-registration-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117942 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-system-cni-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.117951 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:51:03.617932631 +0000 UTC m=+2.055541433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.117956 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-system-cni-dir\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.312919 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118000 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-cnibin\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-os-release\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118036 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-system-cni-dir\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118056 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmhr6\" (UniqueName: \"kubernetes.io/projected/8fdeba2b-e439-4ece-82fb-d7f845764793-kube-api-access-mmhr6\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118070 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-cnibin\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118091 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-cni-bin\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118113 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-os-release\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118137 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-cni-bin\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118204 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-conf-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118268 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf75f52c-b77a-4a9a-8b02-7e823c68380a-dbus\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118324 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118294 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-conf-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118357 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-cni-binary-copy\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118361 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.313561 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118395 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fdeba2b-e439-4ece-82fb-d7f845764793-host-slash\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118409 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf75f52c-b77a-4a9a-8b02-7e823c68380a-dbus\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118415 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-cni-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118416 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118441 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8be98fb-fca8-4048-b97e-b042af16da82-cni-binary-copy\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118452 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fdeba2b-e439-4ece-82fb-d7f845764793-host-slash\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118458 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-kubelet\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118474 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-hostroot\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118475 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-multus-cni-dir\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-multus-certs\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118506 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-etc-kubernetes\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118512 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-var-lib-kubelet\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118525 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-device-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118555 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-etc-kubernetes\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314055 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118569 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-hostroot\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118579 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-sys-fs\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118565 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01aac545-69a2-4eda-97dc-e2a200140a51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118623 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8fdeba2b-e439-4ece-82fb-d7f845764793-iptables-alerter-script\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118638 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-device-dir\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118647 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-cnibin\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118661 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-host-run-multus-certs\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118656 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b85a86b-ed33-479c-a7f8-f07197ea822d-sys-fs\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118676 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f8be98fb-fca8-4048-b97e-b042af16da82-multus-daemon-config\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8hn4\" (UniqueName: \"kubernetes.io/projected/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-kube-api-access-n8hn4\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118735 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5707ea40-64ff-4bf3-b355-32b3d29f80f8-agent-certs\") pod \"konnectivity-agent-858gf\" (UID: \"5707ea40-64ff-4bf3-b355-32b3d29f80f8\") " pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.118705 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8be98fb-fca8-4048-b97e-b042af16da82-cnibin\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.119242 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f8be98fb-fca8-4048-b97e-b042af16da82-multus-daemon-config\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.119252 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8fdeba2b-e439-4ece-82fb-d7f845764793-iptables-alerter-script\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.119696 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8be98fb-fca8-4048-b97e-b042af16da82-cni-binary-copy\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.314639 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.120594 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01aac545-69a2-4eda-97dc-e2a200140a51-cni-binary-copy\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.121170 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5707ea40-64ff-4bf3-b355-32b3d29f80f8-agent-certs\") pod \"konnectivity-agent-858gf\" (UID: \"5707ea40-64ff-4bf3-b355-32b3d29f80f8\") " pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.130190 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skmjb\" (UniqueName: \"kubernetes.io/projected/4b85a86b-ed33-479c-a7f8-f07197ea822d-kube-api-access-skmjb\") pod \"aws-ebs-csi-driver-node-jgdwv\" (UID: \"4b85a86b-ed33-479c-a7f8-f07197ea822d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.138876 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.138977 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.138999 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.139211 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:03.63916337 +0000 UTC m=+2.076772154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.141329 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8hn4\" (UniqueName: \"kubernetes.io/projected/4796b9f6-2024-4e6c-a008-8f3060ea1b4e-kube-api-access-n8hn4\") pod \"node-ca-fl65z\" (UID: \"4796b9f6-2024-4e6c-a008-8f3060ea1b4e\") " pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.141432 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmhr6\" (UniqueName: \"kubernetes.io/projected/8fdeba2b-e439-4ece-82fb-d7f845764793-kube-api-access-mmhr6\") pod \"iptables-alerter-9nbnd\" (UID: \"8fdeba2b-e439-4ece-82fb-d7f845764793\") " pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.147873 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtw2j\" (UniqueName: \"kubernetes.io/projected/b731e8a7-a469-4d64-9ac5-14ced255483d-kube-api-access-qtw2j\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.154189 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngzdf\" (UniqueName: \"kubernetes.io/projected/01aac545-69a2-4eda-97dc-e2a200140a51-kube-api-access-ngzdf\") pod \"multus-additional-cni-plugins-26sc2\" (UID: \"01aac545-69a2-4eda-97dc-e2a200140a51\") " pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.162797 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81cd5de19bf3e7dbbe6adfa180df0e0a.slice/crio-943fdab84b9e77ae05ce46df57d17e18fd8eb29c9d5a3f935d169a94d61d3510 WatchSource:0}: Error finding container 943fdab84b9e77ae05ce46df57d17e18fd8eb29c9d5a3f935d169a94d61d3510: Status 404 returned error can't find the container with id 943fdab84b9e77ae05ce46df57d17e18fd8eb29c9d5a3f935d169a94d61d3510 Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.166882 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a36e4b47671bef90a84d11574ce86d.slice/crio-9be4a1767ccff17f61e2f79ec17399822fe7ccd5081aa41294521903fb4a3ec3 WatchSource:0}: Error finding container 9be4a1767ccff17f61e2f79ec17399822fe7ccd5081aa41294521903fb4a3ec3: Status 404 returned error can't find the container with id 9be4a1767ccff17f61e2f79ec17399822fe7ccd5081aa41294521903fb4a3ec3 Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.169059 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 13:51:03.315336 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.170074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j6m2\" (UniqueName: \"kubernetes.io/projected/f8be98fb-fca8-4048-b97e-b042af16da82-kube-api-access-2j6m2\") pod \"multus-k4k64\" (UID: \"f8be98fb-fca8-4048-b97e-b042af16da82\") " pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.219258 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.230181 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039fd736_4e61_418b_9391_1d10f7401fc0.slice/crio-559cac7dc3f0213341ea9f16a40adfa9be7d1b2c59ad9a640c0908ebb0de01eb WatchSource:0}: Error finding container 559cac7dc3f0213341ea9f16a40adfa9be7d1b2c59ad9a640c0908ebb0de01eb: Status 404 returned error can't find the container with id 559cac7dc3f0213341ea9f16a40adfa9be7d1b2c59ad9a640c0908ebb0de01eb Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.252095 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.263606 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba93ea4_b1a2_4de0_90f0_8065fb23c014.slice/crio-1d57ff0d5005a009ca6f0115ae46e4b4692f15ab72112e7ba0a4708e326a86d2 WatchSource:0}: Error finding container 1d57ff0d5005a009ca6f0115ae46e4b4692f15ab72112e7ba0a4708e326a86d2: Status 404 returned error can't find the container with id 1d57ff0d5005a009ca6f0115ae46e4b4692f15ab72112e7ba0a4708e326a86d2 Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.273839 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9whzn" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.282277 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9618dd0e_33ad_4458_88d6_7442904afcf4.slice/crio-08eac66a63454a7ac7ff9c8f062a48ee13258b92c80e06b8e7bcea2dc2472f7f WatchSource:0}: Error finding container 08eac66a63454a7ac7ff9c8f062a48ee13258b92c80e06b8e7bcea2dc2472f7f: Status 404 returned error can't find the container with id 08eac66a63454a7ac7ff9c8f062a48ee13258b92c80e06b8e7bcea2dc2472f7f Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.291444 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k4k64" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.300024 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.300639 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8be98fb_fca8_4048_b97e_b042af16da82.slice/crio-e14f6253446eb8dc83d59dc6e3d8db8adf77992b55fc3a082e43ef8dbf26ee81 WatchSource:0}: Error finding container e14f6253446eb8dc83d59dc6e3d8db8adf77992b55fc3a082e43ef8dbf26ee81: Status 404 returned error can't find the container with id e14f6253446eb8dc83d59dc6e3d8db8adf77992b55fc3a082e43ef8dbf26ee81 Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.313315 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" Jul 15 13:51:03.316117 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.314804 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5707ea40_64ff_4bf3_b355_32b3d29f80f8.slice/crio-e0e1b3cff4dd130d4e598802eef33c2ed994591b911dd63bc310048cf77fedec WatchSource:0}: Error finding container e0e1b3cff4dd130d4e598802eef33c2ed994591b911dd63bc310048cf77fedec: Status 404 returned error can't find the container with id e0e1b3cff4dd130d4e598802eef33c2ed994591b911dd63bc310048cf77fedec Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.338639 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fl65z" Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.344968 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-26sc2" Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.351136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9nbnd" Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:03.353727 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4796b9f6_2024_4e6c_a008_8f3060ea1b4e.slice/crio-214617a84cf7c1f072041ab2b898f30619756dcea22d2346e49fa405aa60f3ff WatchSource:0}: Error finding container 214617a84cf7c1f072041ab2b898f30619756dcea22d2346e49fa405aa60f3ff: Status 404 returned error can't find the container with id 214617a84cf7c1f072041ab2b898f30619756dcea22d2346e49fa405aa60f3ff Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.362910 2584 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-9nbnd_openshift-network-operator(8fdeba2b-e439-4ece-82fb-d7f845764793): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.362938 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-9nbnd" podUID="8fdeba2b-e439-4ece-82fb-d7f845764793" Jul 15 13:51:03.515753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.395528 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 13:51:03.679682 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.622787 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:03.679682 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.622842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:03.679682 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.623002 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:03.679682 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.623063 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:51:04.623044836 +0000 UTC m=+3.060653609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:03.679682 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.623060 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:03.679682 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.623143 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:04.623124265 +0000 UTC m=+3.060733043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:03.764040 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.723620 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:03.764040 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.723867 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:03.764040 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.723902 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:03.764040 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.723919 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:03.764040 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:03.724014 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:04.723987857 +0000 UTC m=+3.161596637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:03.943265 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.891485 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 13:51:04.025969 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.993091 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 13:46:03 +0000 UTC" deadline="2028-03-10 23:41:16.727404391 +0000 UTC" Jul 15 13:51:04.025969 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:03.993154 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14505h50m12.734255323s" Jul 15 13:51:04.025969 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.021708 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9nbnd" event={"ID":"8fdeba2b-e439-4ece-82fb-d7f845764793","Type":"ContainerStarted","Data":"aaa6e53bfacf178e080cb02268d43fc28c27c43aa07ad4398c508827911dbf18"} Jul 15 13:51:04.025969 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.024627 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:81e3f2576c5145aebf33f0a2d300aef8b70217b00c31101b55ed112e7e2d0087\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-9nbnd" podUID="8fdeba2b-e439-4ece-82fb-d7f845764793" Jul 15 13:51:04.042139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.029742 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerStarted","Data":"277cd93fcaaa922f48403aa59c1c280ffef30edd57ee7716ef2afaec3dd3c8f5"} Jul 15 13:51:04.042139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.031907 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fl65z" event={"ID":"4796b9f6-2024-4e6c-a008-8f3060ea1b4e","Type":"ContainerStarted","Data":"214617a84cf7c1f072041ab2b898f30619756dcea22d2346e49fa405aa60f3ff"} Jul 15 13:51:04.042139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.039051 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" event={"ID":"4b85a86b-ed33-479c-a7f8-f07197ea822d","Type":"ContainerStarted","Data":"cd453ad7925433b872f44cfdd41bd9be226d06c3d7abf5142656260007c76e1d"} Jul 15 13:51:04.116618 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.051340 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-858gf" event={"ID":"5707ea40-64ff-4bf3-b355-32b3d29f80f8","Type":"ContainerStarted","Data":"e0e1b3cff4dd130d4e598802eef33c2ed994591b911dd63bc310048cf77fedec"} Jul 15 13:51:04.116618 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.063771 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9whzn" event={"ID":"9618dd0e-33ad-4458-88d6-7442904afcf4","Type":"ContainerStarted","Data":"08eac66a63454a7ac7ff9c8f062a48ee13258b92c80e06b8e7bcea2dc2472f7f"} Jul 15 13:51:04.116618 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.073719 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" event={"ID":"dba93ea4-b1a2-4de0-90f0-8065fb23c014","Type":"ContainerStarted","Data":"1d57ff0d5005a009ca6f0115ae46e4b4692f15ab72112e7ba0a4708e326a86d2"} Jul 15 13:51:04.116618 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.077905 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"559cac7dc3f0213341ea9f16a40adfa9be7d1b2c59ad9a640c0908ebb0de01eb"} Jul 15 13:51:04.116618 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.099103 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k4k64" event={"ID":"f8be98fb-fca8-4048-b97e-b042af16da82","Type":"ContainerStarted","Data":"e14f6253446eb8dc83d59dc6e3d8db8adf77992b55fc3a082e43ef8dbf26ee81"} Jul 15 13:51:04.125636 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.125561 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" event={"ID":"72a36e4b47671bef90a84d11574ce86d","Type":"ContainerStarted","Data":"9be4a1767ccff17f61e2f79ec17399822fe7ccd5081aa41294521903fb4a3ec3"} Jul 15 13:51:04.147758 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.147722 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" event={"ID":"81cd5de19bf3e7dbbe6adfa180df0e0a","Type":"ContainerStarted","Data":"943fdab84b9e77ae05ce46df57d17e18fd8eb29c9d5a3f935d169a94d61d3510"} Jul 15 13:51:04.641295 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.640472 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:04.641295 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.640535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:04.641295 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.640687 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:04.641295 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.640753 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:51:06.640734081 +0000 UTC m=+5.078342859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:04.641295 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.641196 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:04.641295 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.641257 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:06.641239783 +0000 UTC m=+5.078848579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:04.742451 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.741791 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:04.742451 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.741969 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:04.742451 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.741989 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:04.742451 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.742003 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:04.742451 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:04.742061 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:06.742043255 +0000 UTC m=+5.179652041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:04.994166 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.994062 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 13:46:03 +0000 UTC" deadline="2028-03-04 01:12:43.353857562 +0000 UTC" Jul 15 13:51:04.994166 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:04.994117 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14339h21m38.359746749s" Jul 15 13:51:05.001534 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:05.001504 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:05.001700 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:05.001642 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:05.002085 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:05.002064 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:05.002201 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:05.002178 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:05.002264 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:05.002252 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:05.002344 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:05.002326 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:06.658484 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:06.657339 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:06.658484 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:06.657401 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:06.658484 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.657576 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:06.658484 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.657646 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:51:10.657626116 +0000 UTC m=+9.095234893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:06.658484 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.658065 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:06.658484 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.658128 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:10.658110077 +0000 UTC m=+9.095718849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:06.759115 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:06.758274 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:06.759115 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.758523 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:06.759115 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.758547 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:06.759115 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.758562 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:06.759115 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:06.758624 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:10.758606134 +0000 UTC m=+9.196214911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:07.001769 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:07.001654 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:07.001769 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:07.001709 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:07.001769 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:07.001709 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:07.002068 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:07.001824 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:07.002068 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:07.001928 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:07.002068 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:07.002021 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:09.001893 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:09.001668 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:09.001893 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:09.001797 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:09.002394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:09.001958 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:09.002394 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:09.002042 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:09.002394 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:09.002082 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:09.002394 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:09.002136 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:10.691441 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:10.691397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:10.691916 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:10.691461 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:10.691916 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.691671 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:10.691916 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.691739 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:51:18.691716626 +0000 UTC m=+17.129325396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:10.692179 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.692157 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:10.692256 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.692216 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:18.692201914 +0000 UTC m=+17.129810700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:10.792501 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:10.792235 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:10.792501 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.792502 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:10.792732 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.792525 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:10.792732 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.792538 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:10.792732 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:10.792600 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:18.792582647 +0000 UTC m=+17.230191428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:11.001817 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:11.001741 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:11.001983 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:11.001845 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:11.001983 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:11.001895 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:11.001983 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:11.001954 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:11.002153 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:11.001988 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:11.002153 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:11.002038 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:13.002088 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:13.001610 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:13.002088 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:13.001724 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:13.002088 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:13.001735 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:13.002088 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:13.001761 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:13.002088 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:13.001864 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:13.002088 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:13.001949 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:15.001927 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:15.001662 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:15.002379 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:15.001702 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:15.002379 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:15.002027 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:15.002379 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:15.001702 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:15.002379 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:15.002132 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:15.002379 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:15.002243 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:17.002128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:17.002085 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:17.002128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:17.002126 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:17.002651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:17.002126 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:17.002651 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:17.002210 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:17.002651 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:17.002326 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:17.002651 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:17.002412 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:18.749802 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:18.749730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:18.749802 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:18.749776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:18.750218 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.749869 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:18.750218 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.749885 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:18.750218 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.749918 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:51:34.749904592 +0000 UTC m=+33.187513362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:18.750218 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.749953 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:34.749932633 +0000 UTC m=+33.187541403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:18.850520 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:18.850476 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:18.850686 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.850630 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:18.850686 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.850659 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:18.850686 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.850672 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:18.850868 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:18.850752 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:34.850731612 +0000 UTC m=+33.288340386 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:19.001701 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:19.001619 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:19.001701 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:19.001662 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:19.001930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:19.001634 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:19.001930 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:19.001749 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:19.001930 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:19.001846 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:19.002074 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:19.001945 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:21.001755 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:21.001695 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:21.001755 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:21.001742 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:21.002193 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:21.001823 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:21.002193 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:21.001841 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:21.002193 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:21.001909 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:21.002193 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:21.001978 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:23.002017 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:23.001978 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:23.002488 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:23.001978 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:23.002488 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:23.001978 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:23.002488 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:23.002234 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:23.002488 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:23.002111 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:23.002488 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:23.002287 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:24.193580 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.193333 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k4k64" event={"ID":"f8be98fb-fca8-4048-b97e-b042af16da82","Type":"ContainerStarted","Data":"c497ac22e86a2939f118ad1a148c91ee10824a6da7b0165b9cab4d64e23b4d45"} Jul 15 13:51:24.195187 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.194947 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" event={"ID":"72a36e4b47671bef90a84d11574ce86d","Type":"ContainerStarted","Data":"204686ed56eaff9d1376c060747a6c1530bcc27d76787ff518eb4425aa49341b"} Jul 15 13:51:24.196399 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.196374 2584 generic.go:356] "Generic (PLEG): container finished" podID="81cd5de19bf3e7dbbe6adfa180df0e0a" containerID="253eeb88167c78336445603749df79eaac9d081c7e8dca31ddda76a049f32055" exitCode=0 Jul 15 13:51:24.196505 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.196442 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" event={"ID":"81cd5de19bf3e7dbbe6adfa180df0e0a","Type":"ContainerDied","Data":"253eeb88167c78336445603749df79eaac9d081c7e8dca31ddda76a049f32055"} Jul 15 13:51:24.197901 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.197877 2584 generic.go:356] "Generic (PLEG): container finished" podID="01aac545-69a2-4eda-97dc-e2a200140a51" containerID="cae92d39b76b1b486ec2bc4d0900a43b3734b7c2f7fe2924262d8b90ef0c2288" exitCode=0 Jul 15 13:51:24.198010 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.197955 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerDied","Data":"cae92d39b76b1b486ec2bc4d0900a43b3734b7c2f7fe2924262d8b90ef0c2288"} Jul 15 13:51:24.199562 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.199537 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fl65z" event={"ID":"4796b9f6-2024-4e6c-a008-8f3060ea1b4e","Type":"ContainerStarted","Data":"207f39d5ff27ea0d921345f8e9b167ca11ec59cb640eff4fff57694dc0e5b70a"} Jul 15 13:51:24.201158 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.201124 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" event={"ID":"4b85a86b-ed33-479c-a7f8-f07197ea822d","Type":"ContainerStarted","Data":"d6a09c55b6766e22ef292f1ac8030d7e5da180e2e72abd90193803c64a698ccb"} Jul 15 13:51:24.202418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.202399 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-858gf" event={"ID":"5707ea40-64ff-4bf3-b355-32b3d29f80f8","Type":"ContainerStarted","Data":"471ff1fc9c6f1096eb927d393134e3395a696ef3b53325e93ac6292c364d4a5a"} Jul 15 13:51:24.203759 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.203732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9whzn" event={"ID":"9618dd0e-33ad-4458-88d6-7442904afcf4","Type":"ContainerStarted","Data":"fefb1c4d35a7dce65afaae7622313524ec4344dbbbdd855319d95cd47fe8d0ed"} Jul 15 13:51:24.205606 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.205586 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" event={"ID":"dba93ea4-b1a2-4de0-90f0-8065fb23c014","Type":"ContainerStarted","Data":"fff81026751c718af6bfdb6f6d490d9dc2556982b4006c5f6e973df2cdad646f"} Jul 15 13:51:24.207885 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.207870 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:51:24.208221 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.208205 2584 generic.go:356] "Generic (PLEG): container finished" podID="039fd736-4e61-418b-9391-1d10f7401fc0" containerID="c4a1a14466ebfe78a346f5350531b1294af41ed773926036a5826bbdbb51535a" exitCode=1 Jul 15 13:51:24.208284 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.208230 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"e059b30da57568ce5d4fb737ffab54f6efe54e7044a99d98e0fe94b7590bd030"} Jul 15 13:51:24.208284 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.208279 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"4800497fc7b071a3f0f3d300550d795757fa125df854b14b074bb94fa678d6c1"} Jul 15 13:51:24.208365 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.208328 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"47d5108b1bf284e6af2d70a8e87621849b13d1a1fedc9460a8642e906750fc37"} Jul 15 13:51:24.208365 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.208357 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerDied","Data":"c4a1a14466ebfe78a346f5350531b1294af41ed773926036a5826bbdbb51535a"} Jul 15 13:51:24.208424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.208384 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"d374a1a72847ebe8c6347f05753e693e59d231e21b9999b8e0d814013c904f98"} Jul 15 13:51:24.215674 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.215623 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k4k64" podStartSLOduration=2.463956658 podStartE2EDuration="22.215596953s" totalImagesPullingTime="19.751640295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:24.215576477 +0000 UTC m=+22.653185263" watchObservedRunningTime="2026-07-15 13:51:24.215596953 +0000 UTC m=+22.653205723" Jul 15 13:51:24.254152 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.254005 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-21.ec2.internal" podStartSLOduration=21.253987429 podStartE2EDuration="21.253987429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:24.253437824 +0000 UTC m=+22.691046621" watchObservedRunningTime="2026-07-15 13:51:24.253987429 +0000 UTC m=+22.691596223" Jul 15 13:51:24.290406 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.290364 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fl65z" podStartSLOduration=2.626659158 podStartE2EDuration="22.290350135s" totalImagesPullingTime="19.663690977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:24.289867009 +0000 UTC m=+22.727475799" watchObservedRunningTime="2026-07-15 13:51:24.290350135 +0000 UTC m=+22.727958926" Jul 15 13:51:24.308387 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.308347 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pqv2j" podStartSLOduration=2.552391351 podStartE2EDuration="22.30833334s" totalImagesPullingTime="19.755941989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:24.307841328 +0000 UTC m=+22.745450120" watchObservedRunningTime="2026-07-15 13:51:24.30833334 +0000 UTC m=+22.745942131" Jul 15 13:51:24.330146 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:24.330098 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9whzn" podStartSLOduration=2.595280657 podStartE2EDuration="22.330084803s" totalImagesPullingTime="19.734804146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:24.329752331 +0000 UTC m=+22.767361123" watchObservedRunningTime="2026-07-15 13:51:24.330084803 +0000 UTC m=+22.767693594" Jul 15 13:51:25.002059 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.001966 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:25.002059 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.002005 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:25.002059 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.002004 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:25.002359 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:25.002126 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:25.002359 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:25.002199 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:25.002359 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:25.002282 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:25.213611 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.213584 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:51:25.214144 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.214019 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"29b1aab748e66be57c73afce6b34330a28461d30c611d780ad2eadd58ce3eba8"} Jul 15 13:51:25.215977 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.215927 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" event={"ID":"81cd5de19bf3e7dbbe6adfa180df0e0a","Type":"ContainerStarted","Data":"264d516b20d79d8fcd8d84a07350a89c4abf1420c0c0d9a5a39fd031adc23537"} Jul 15 13:51:25.217677 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.217643 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9nbnd" event={"ID":"8fdeba2b-e439-4ece-82fb-d7f845764793","Type":"ContainerStarted","Data":"76672d7bd525d2866719cfa1acbaa0531dad2799fd61618f1d9a40d462cbb792"} Jul 15 13:51:25.235412 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.235359 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-858gf" podStartSLOduration=3.532627917 podStartE2EDuration="23.23534081s" totalImagesPullingTime="19.702712893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:24.360523683 +0000 UTC m=+22.798132475" watchObservedRunningTime="2026-07-15 13:51:25.23534081 +0000 UTC m=+23.672949603" Jul 15 13:51:25.236334 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.236278 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-21.ec2.internal" podStartSLOduration=22.236269195 podStartE2EDuration="22.236269195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:25.235429899 +0000 UTC m=+23.673038695" watchObservedRunningTime="2026-07-15 13:51:25.236269195 +0000 UTC m=+23.673877987" Jul 15 13:51:25.251475 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.251435 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9nbnd" podStartSLOduration=3.270664053 podStartE2EDuration="23.251415051s" totalImagesPullingTime="19.980750998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-15 13:51:25.251052574 +0000 UTC m=+23.688661367" watchObservedRunningTime="2026-07-15 13:51:25.251415051 +0000 UTC m=+23.689023844" Jul 15 13:51:25.441022 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.440986 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 13:51:25.985623 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.985528 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T13:51:25.441005357Z","UUID":"c7520875-bb52-4aea-bdc2-28b4a7d7d2f0","Handler":null,"Name":"","Endpoint":""} Jul 15 13:51:25.987416 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.987387 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 13:51:25.987554 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:25.987445 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 13:51:26.223061 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:26.223004 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" event={"ID":"4b85a86b-ed33-479c-a7f8-f07197ea822d","Type":"ContainerStarted","Data":"53e6c4cf33a2845465502c0e42ad5f07941aa8a7cdedbd94bd462e446fe4b797"} Jul 15 13:51:27.002212 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.002172 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:27.002424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.002172 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:27.002424 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:27.002313 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:27.002424 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:27.002409 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:27.002578 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.002176 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:27.002578 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:27.002535 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:27.228873 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.228844 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:51:27.229279 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.229162 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"8c5b3b3b8655b94b06246f30d55b322650a5ee4de4dd79be895b4ee47ab3c2b7"} Jul 15 13:51:27.693482 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.693446 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:27.698827 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:27.694620 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:28.231446 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:28.231397 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:28.232037 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:28.232022 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-858gf" Jul 15 13:51:29.001551 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.001377 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:29.001688 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.001392 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:29.001688 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:29.001622 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:29.001762 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.001396 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:29.001762 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:29.001695 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:29.001829 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:29.001790 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:29.235216 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.235188 2584 generic.go:356] "Generic (PLEG): container finished" podID="01aac545-69a2-4eda-97dc-e2a200140a51" containerID="cabd1430a55334a7f36a8faad1080f2d8763fbb4b138d8aab5f214554acec285" exitCode=0 Jul 15 13:51:29.236093 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.235266 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerDied","Data":"cabd1430a55334a7f36a8faad1080f2d8763fbb4b138d8aab5f214554acec285"} Jul 15 13:51:29.237431 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.237410 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" event={"ID":"4b85a86b-ed33-479c-a7f8-f07197ea822d","Type":"ContainerStarted","Data":"1d070ef4c0083a535ec22f42155da0cc42489b7809ac24b3572a40a549f3bc7d"} Jul 15 13:51:29.242613 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.242593 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:51:29.243125 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.243102 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"5774e41b0dd2305aca3db611ab6db018aa24b15f2b2c977bc1d99a1f2d2e8997"} Jul 15 13:51:29.243277 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.243257 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:29.243560 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.243545 2584 scope.go:117] "RemoveContainer" containerID="c4a1a14466ebfe78a346f5350531b1294af41ed773926036a5826bbdbb51535a" Jul 15 13:51:29.260529 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.260511 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:29.278705 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:29.278663 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgdwv" podStartSLOduration=3.636688747 podStartE2EDuration="27.278649804s" totalImagesPullingTime="23.641961057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:29.278062712 +0000 UTC m=+27.715671503" watchObservedRunningTime="2026-07-15 13:51:29.278649804 +0000 UTC m=+27.716258634" Jul 15 13:51:30.249436 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.249185 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:51:30.249886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.249758 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" event={"ID":"039fd736-4e61-418b-9391-1d10f7401fc0","Type":"ContainerStarted","Data":"4390ac399929c3d907c9b6f47744365f76c6226affca2e8624c33bf09dcf69cc"} Jul 15 13:51:30.250422 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.250402 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:30.250534 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.250468 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:30.265680 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.265654 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:51:30.314043 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.314000 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" podStartSLOduration=7.98966049 podStartE2EDuration="28.313984414s" totalImagesPullingTime="20.324323924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:30.284023928 +0000 UTC m=+28.721632723" watchObservedRunningTime="2026-07-15 13:51:30.313984414 +0000 UTC m=+28.751593206" Jul 15 13:51:30.524539 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.524512 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5qvnw"] Jul 15 13:51:30.524652 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.524641 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:30.524771 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:30.524747 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:30.527691 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.527671 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t7tnx"] Jul 15 13:51:30.527804 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.527774 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:30.527867 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:30.527855 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:30.528379 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.528355 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ncnc9"] Jul 15 13:51:30.528484 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:30.528445 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:30.528562 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:30.528542 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:31.254155 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:31.254060 2584 generic.go:356] "Generic (PLEG): container finished" podID="01aac545-69a2-4eda-97dc-e2a200140a51" containerID="ad48cfebed39ab8650f58c6c04b70f69f08d9985d133c8c1707b06aff0e5e2c2" exitCode=0 Jul 15 13:51:31.254744 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:31.254153 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerDied","Data":"ad48cfebed39ab8650f58c6c04b70f69f08d9985d133c8c1707b06aff0e5e2c2"} Jul 15 13:51:32.002096 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:32.002063 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:32.002265 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:32.002160 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:32.002265 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:32.002167 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:32.002388 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:32.002268 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:32.002388 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:32.002276 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:32.002388 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:32.002369 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:33.261586 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:33.261415 2584 generic.go:356] "Generic (PLEG): container finished" podID="01aac545-69a2-4eda-97dc-e2a200140a51" containerID="ac0a3a45109f49ecb1a282a3013b157321b47b52bc30882072b6c28129c941f4" exitCode=0 Jul 15 13:51:33.261956 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:33.261490 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerDied","Data":"ac0a3a45109f49ecb1a282a3013b157321b47b52bc30882072b6c28129c941f4"} Jul 15 13:51:34.004798 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:34.004769 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:34.005017 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:34.004773 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:34.005017 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.004869 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:34.005017 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:34.004894 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:34.005017 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.005006 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:34.005265 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.005095 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:34.772089 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:34.772050 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:34.772531 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:34.772103 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:34.772531 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.772238 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:34.772531 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.772240 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:34.772531 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.772318 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs podName:b731e8a7-a469-4d64-9ac5-14ced255483d nodeName:}" failed. No retries permitted until 2026-07-15 13:52:06.772280255 +0000 UTC m=+65.209889029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs") pod "network-metrics-daemon-ncnc9" (UID: "b731e8a7-a469-4d64-9ac5-14ced255483d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 13:51:34.772531 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.772406 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret podName:cf75f52c-b77a-4a9a-8b02-7e823c68380a nodeName:}" failed. No retries permitted until 2026-07-15 13:52:06.772386388 +0000 UTC m=+65.209995163 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret") pod "global-pull-secret-syncer-t7tnx" (UID: "cf75f52c-b77a-4a9a-8b02-7e823c68380a") : object "kube-system"/"original-pull-secret" not registered Jul 15 13:51:34.873401 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:34.873371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:34.873578 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.873534 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 13:51:34.873578 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.873553 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 13:51:34.873578 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.873564 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kqd8j for pod openshift-network-diagnostics/network-check-target-5qvnw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:34.873741 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:34.873618 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j podName:3be71432-c1ef-419b-a4c1-645ea7cb8aa8 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:06.873600603 +0000 UTC m=+65.311209391 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqd8j" (UniqueName: "kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j") pod "network-check-target-5qvnw" (UID: "3be71432-c1ef-419b-a4c1-645ea7cb8aa8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 13:51:36.001645 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.001609 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:36.002234 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.001725 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qvnw" podUID="3be71432-c1ef-419b-a4c1-645ea7cb8aa8" Jul 15 13:51:36.002234 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.001609 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:36.002234 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.001609 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:36.002234 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.001822 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t7tnx" podUID="cf75f52c-b77a-4a9a-8b02-7e823c68380a" Jul 15 13:51:36.002234 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.001916 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ncnc9" podUID="b731e8a7-a469-4d64-9ac5-14ced255483d" Jul 15 13:51:36.353849 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.353819 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-21.ec2.internal" event="NodeReady" Jul 15 13:51:36.354027 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.353974 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 13:51:36.402859 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.402828 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-66c5645f8-m26q2"] Jul 15 13:51:36.444190 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.444140 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-8pcv4"] Jul 15 13:51:36.444498 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.444474 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.450497 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.450469 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 13:51:36.459152 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.459134 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 13:51:36.459672 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.459649 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 13:51:36.459779 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.459726 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.459877 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.459841 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 13:51:36.459986 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.459967 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jfsg6\"" Jul 15 13:51:36.460053 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.459850 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.460121 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.460102 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw"] Jul 15 13:51:36.460196 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.460184 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.464546 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.464522 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 13:51:36.464645 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.464579 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.464645 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.464612 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.464645 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.464640 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 13:51:36.465658 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.465638 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-574pg\"" Jul 15 13:51:36.478195 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.478170 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx"] Jul 15 13:51:36.478323 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.478306 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.481400 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.481381 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.481495 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.481399 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-srk4z\"" Jul 15 13:51:36.481602 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.481583 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 13:51:36.481693 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.481651 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.502353 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.502332 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-678f7cb99d-cd67v"] Jul 15 13:51:36.502480 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.502452 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:36.514226 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.514198 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 13:51:36.515765 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.515745 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 13:51:36.515866 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.515772 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.516533 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.516506 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-b2x5t\"" Jul 15 13:51:36.516533 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.516520 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.518934 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.518567 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7ddh9"] Jul 15 13:51:36.519100 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.519082 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.551329 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.550984 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 13:51:36.551329 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.551260 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 13:51:36.551559 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.551489 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 13:51:36.551656 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.551639 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 13:51:36.551829 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.551808 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kq22p\"" Jul 15 13:51:36.554022 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.553475 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb"] Jul 15 13:51:36.554256 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.554222 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.559251 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.559228 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.559434 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.559415 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 13:51:36.559552 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.559477 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 13:51:36.559632 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.559615 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rvldv\"" Jul 15 13:51:36.559844 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.559827 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.560058 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.560035 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 13:51:36.564326 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.564289 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 13:51:36.576469 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576446 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-8pcv4"] Jul 15 13:51:36.576585 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576487 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw"] Jul 15 13:51:36.576585 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576507 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66c5645f8-m26q2"] Jul 15 13:51:36.576585 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576526 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-678f7cb99d-cd67v"] Jul 15 13:51:36.576585 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576541 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7ddh9"] Jul 15 13:51:36.576585 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576557 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx"] Jul 15 13:51:36.576585 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576581 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn"] Jul 15 13:51:36.576879 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.576610 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" Jul 15 13:51:36.579689 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.579667 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.579791 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.579698 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.579791 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.579713 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-n5jrt\"" Jul 15 13:51:36.586008 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.585976 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhfb\" (UniqueName: \"kubernetes.io/projected/9f828c0e-e03b-4683-8405-028251958c94-kube-api-access-xvhfb\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.586103 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/516d2f7f-144e-4862-9e00-fc044804030a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.586103 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586071 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzc98\" (UniqueName: \"kubernetes.io/projected/516d2f7f-144e-4862-9e00-fc044804030a-kube-api-access-hzc98\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.586217 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586109 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4q7\" (UniqueName: \"kubernetes.io/projected/864878a2-8862-420c-886d-0d7b2cf462c9-kube-api-access-jw4q7\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.586217 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586135 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f828c0e-e03b-4683-8405-028251958c94-config\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.586217 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586150 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f828c0e-e03b-4683-8405-028251958c94-serving-cert\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.586217 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586166 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f828c0e-e03b-4683-8405-028251958c94-trusted-ca\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.586217 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-default-certificate\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.586493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586242 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.586493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586308 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:36.586493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586340 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr2rf\" (UniqueName: \"kubernetes.io/projected/9f378cf9-965b-4be4-a301-ec30d18361ce-kube-api-access-fr2rf\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:36.586493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586373 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.586493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586395 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.586493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.586415 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-stats-auth\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.600586 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.600566 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl"] Jul 15 13:51:36.600726 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.600708 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.604105 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.604062 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.604316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.604276 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 13:51:36.604409 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.604345 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lwcvm\"" Jul 15 13:51:36.604409 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.604350 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 13:51:36.604546 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.604527 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.618766 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.618748 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb"] Jul 15 13:51:36.618884 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.618779 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zmd4q"] Jul 15 13:51:36.618884 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.618873 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.622753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.622291 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 13:51:36.622753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.622470 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.622753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.622564 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4z5qm\"" Jul 15 13:51:36.622753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.622716 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.624155 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.624078 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 13:51:36.642829 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.642810 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl"] Jul 15 13:51:36.642947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.642876 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zmd4q"] Jul 15 13:51:36.642947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.642894 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn"] Jul 15 13:51:36.642947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.642916 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dsffv"] Jul 15 13:51:36.643106 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.643021 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.646819 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.646799 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.647332 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.647312 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.647545 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.647531 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-75k4w\"" Jul 15 13:51:36.647834 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.647819 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 13:51:36.666870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.666849 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f"] Jul 15 13:51:36.666984 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.666965 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.669833 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.669813 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 13:51:36.669902 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.669861 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2df9f\"" Jul 15 13:51:36.670086 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.670061 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 13:51:36.687431 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687404 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47accf41-e456-4b19-9cb2-edb9c0369ac2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.687546 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687441 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:36.687546 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg79g\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-kube-api-access-pg79g\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.687546 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687486 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4q7\" (UniqueName: \"kubernetes.io/projected/864878a2-8862-420c-886d-0d7b2cf462c9-kube-api-access-jw4q7\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.687546 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687526 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-trusted-ca\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f828c0e-e03b-4683-8405-028251958c94-trusted-ca\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.687573 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687590 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4fde022e-b51b-49fe-a86c-1e142a744c5c-ca-trust-extracted\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687613 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-bound-sa-token\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687637 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.687664 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls podName:9f378cf9-965b-4be4-a301-ec30d18361ce nodeName:}" failed. No retries permitted until 2026-07-15 13:51:37.187643431 +0000 UTC m=+35.625252201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls") pod "cluster-samples-operator-65db858894-8lfmx" (UID: "9f378cf9-965b-4be4-a301-ec30d18361ce") : secret "samples-operator-tls" not found Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-installation-pull-secrets\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.687750 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687747 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687789 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687815 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-stats-auth\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.687818 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687870 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.687857 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.687915 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:37.187902665 +0000 UTC m=+35.625511435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : secret "router-metrics-certs-default" not found Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687928 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhfb\" (UniqueName: \"kubernetes.io/projected/9f828c0e-e03b-4683-8405-028251958c94-kube-api-access-xvhfb\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.687948 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47accf41-e456-4b19-9cb2-edb9c0369ac2-serving-cert\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.687968 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:37.18795011 +0000 UTC m=+35.625558899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : configmap references non-existent config key: service-ca.crt Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.688010 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls podName:516d2f7f-144e-4862-9e00-fc044804030a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:37.187999541 +0000 UTC m=+35.625608312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tmpw" (UID: "516d2f7f-144e-4862-9e00-fc044804030a") : secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:36.688122 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f828c0e-e03b-4683-8405-028251958c94-serving-cert\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688089 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzc98\" (UniqueName: \"kubernetes.io/projected/516d2f7f-144e-4862-9e00-fc044804030a-kube-api-access-hzc98\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688114 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr2rf\" (UniqueName: \"kubernetes.io/projected/9f378cf9-965b-4be4-a301-ec30d18361ce-kube-api-access-fr2rf\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688144 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9k6l\" (UniqueName: \"kubernetes.io/projected/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-kube-api-access-j9k6l\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688172 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47accf41-e456-4b19-9cb2-edb9c0369ac2-tmp\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688233 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f828c0e-e03b-4683-8405-028251958c94-config\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-default-certificate\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688289 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/47accf41-e456-4b19-9cb2-edb9c0369ac2-snapshots\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688358 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47accf41-e456-4b19-9cb2-edb9c0369ac2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688389 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xgkc\" (UniqueName: \"kubernetes.io/projected/47accf41-e456-4b19-9cb2-edb9c0369ac2-kube-api-access-9xgkc\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688417 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-image-registry-private-configuration\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.688708 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688450 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hldw7\" (UniqueName: \"kubernetes.io/projected/d3f3476a-97d2-415c-ab0a-ff9fd002ff50-kube-api-access-hldw7\") pod \"volume-data-source-validator-78495597f6-h2twb\" (UID: \"d3f3476a-97d2-415c-ab0a-ff9fd002ff50\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" Jul 15 13:51:36.689203 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688476 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-certificates\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.689203 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688504 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/516d2f7f-144e-4862-9e00-fc044804030a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.689203 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688546 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f828c0e-e03b-4683-8405-028251958c94-trusted-ca\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.689203 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.688915 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f828c0e-e03b-4683-8405-028251958c94-config\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.689203 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.689191 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/516d2f7f-144e-4862-9e00-fc044804030a-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.691029 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.691007 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dsffv"] Jul 15 13:51:36.691121 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.691053 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f"] Jul 15 13:51:36.691121 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.691115 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" Jul 15 13:51:36.692662 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.692637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f828c0e-e03b-4683-8405-028251958c94-serving-cert\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.692770 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.692751 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-stats-auth\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.692843 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.692751 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-default-certificate\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.694251 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.694232 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vns6c\"" Jul 15 13:51:36.694839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.694813 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 13:51:36.694920 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.694822 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 13:51:36.697131 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.697111 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4q7\" (UniqueName: \"kubernetes.io/projected/864878a2-8862-420c-886d-0d7b2cf462c9-kube-api-access-jw4q7\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:36.701663 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.701623 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr2rf\" (UniqueName: \"kubernetes.io/projected/9f378cf9-965b-4be4-a301-ec30d18361ce-kube-api-access-fr2rf\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:36.703697 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.703670 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhfb\" (UniqueName: \"kubernetes.io/projected/9f828c0e-e03b-4683-8405-028251958c94-kube-api-access-xvhfb\") pod \"console-operator-7866497494-8pcv4\" (UID: \"9f828c0e-e03b-4683-8405-028251958c94\") " pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.703697 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.703685 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzc98\" (UniqueName: \"kubernetes.io/projected/516d2f7f-144e-4862-9e00-fc044804030a-kube-api-access-hzc98\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:36.769108 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.769074 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:36.789022 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.788980 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6pt\" (UniqueName: \"kubernetes.io/projected/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-kube-api-access-pp6pt\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.789022 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789024 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43e414e3-fc9e-40ee-912a-6597bd9d2faf-cert\") pod \"ingress-canary-zmd4q\" (UID: \"43e414e3-fc9e-40ee-912a-6597bd9d2faf\") " pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.789224 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789091 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.789224 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789130 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47accf41-e456-4b19-9cb2-edb9c0369ac2-serving-cert\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.789224 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789158 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-config\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.789224 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9k6l\" (UniqueName: \"kubernetes.io/projected/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-kube-api-access-j9k6l\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789233 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47accf41-e456-4b19-9cb2-edb9c0369ac2-tmp\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789258 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789289 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789334 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/47accf41-e456-4b19-9cb2-edb9c0369ac2-snapshots\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789361 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47accf41-e456-4b19-9cb2-edb9c0369ac2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789389 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xgkc\" (UniqueName: \"kubernetes.io/projected/47accf41-e456-4b19-9cb2-edb9c0369ac2-kube-api-access-9xgkc\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.789424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789415 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-image-registry-private-configuration\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.789756 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789450 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hldw7\" (UniqueName: \"kubernetes.io/projected/d3f3476a-97d2-415c-ab0a-ff9fd002ff50-kube-api-access-hldw7\") pod \"volume-data-source-validator-78495597f6-h2twb\" (UID: \"d3f3476a-97d2-415c-ab0a-ff9fd002ff50\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" Jul 15 13:51:36.789756 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789474 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-certificates\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.789756 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789505 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlr7h\" (UniqueName: \"kubernetes.io/projected/43e414e3-fc9e-40ee-912a-6597bd9d2faf-kube-api-access-dlr7h\") pod \"ingress-canary-zmd4q\" (UID: \"43e414e3-fc9e-40ee-912a-6597bd9d2faf\") " pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.789902 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.789810 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.790205 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790178 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/47accf41-e456-4b19-9cb2-edb9c0369ac2-snapshots\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790625 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47accf41-e456-4b19-9cb2-edb9c0369ac2-tmp\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790641 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47accf41-e456-4b19-9cb2-edb9c0369ac2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790693 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47accf41-e456-4b19-9cb2-edb9c0369ac2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790738 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-serving-cert\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790900 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-config-volume\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-tmp-dir\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790959 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg79g\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-kube-api-access-pg79g\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790966 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-certificates\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.790985 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpnsj\" (UniqueName: \"kubernetes.io/projected/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-kube-api-access-bpnsj\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.791018 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791023 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-trusted-ca\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791051 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mr4z\" (UniqueName: \"kubernetes.io/projected/4d47a0cb-4e26-437d-9aa1-2735e9ace4f9-kube-api-access-8mr4z\") pod \"network-check-source-869dcfcdcd-d7v2f\" (UID: \"4d47a0cb-4e26-437d-9aa1-2735e9ace4f9\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" Jul 15 13:51:36.791582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4fde022e-b51b-49fe-a86c-1e142a744c5c-ca-trust-extracted\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791108 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-bound-sa-token\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791134 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791152 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47accf41-e456-4b19-9cb2-edb9c0369ac2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.791582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791161 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-installation-pull-secrets\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.791863 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.791644 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4fde022e-b51b-49fe-a86c-1e142a744c5c-ca-trust-extracted\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.792183 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.791913 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 13:51:36.792183 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.791934 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678f7cb99d-cd67v: secret "image-registry-tls" not found Jul 15 13:51:36.792183 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.791994 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls podName:4fde022e-b51b-49fe-a86c-1e142a744c5c nodeName:}" failed. No retries permitted until 2026-07-15 13:51:37.291976295 +0000 UTC m=+35.729585078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls") pod "image-registry-678f7cb99d-cd67v" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c") : secret "image-registry-tls" not found Jul 15 13:51:36.792183 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.792167 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-trusted-ca\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.792498 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.792276 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47accf41-e456-4b19-9cb2-edb9c0369ac2-serving-cert\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.792498 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.792353 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-image-registry-private-configuration\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.793024 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.792954 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.794349 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.794327 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-installation-pull-secrets\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.808083 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.808055 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9k6l\" (UniqueName: \"kubernetes.io/projected/54a8e011-19a1-4f5e-a00d-e7f937aab4d2-kube-api-access-j9k6l\") pod \"kube-storage-version-migrator-operator-799cb7f958-85mvn\" (UID: \"54a8e011-19a1-4f5e-a00d-e7f937aab4d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.808191 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.808054 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-bound-sa-token\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.808191 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.808125 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xgkc\" (UniqueName: \"kubernetes.io/projected/47accf41-e456-4b19-9cb2-edb9c0369ac2-kube-api-access-9xgkc\") pod \"insights-operator-68d9dd94cf-7ddh9\" (UID: \"47accf41-e456-4b19-9cb2-edb9c0369ac2\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.809532 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.809509 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hldw7\" (UniqueName: \"kubernetes.io/projected/d3f3476a-97d2-415c-ab0a-ff9fd002ff50-kube-api-access-hldw7\") pod \"volume-data-source-validator-78495597f6-h2twb\" (UID: \"d3f3476a-97d2-415c-ab0a-ff9fd002ff50\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" Jul 15 13:51:36.809891 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.809869 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg79g\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-kube-api-access-pg79g\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:36.866355 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.866261 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" Jul 15 13:51:36.886584 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.886551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" Jul 15 13:51:36.891974 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.891943 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-config\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.892106 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892012 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.892106 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlr7h\" (UniqueName: \"kubernetes.io/projected/43e414e3-fc9e-40ee-912a-6597bd9d2faf-kube-api-access-dlr7h\") pod \"ingress-canary-zmd4q\" (UID: \"43e414e3-fc9e-40ee-912a-6597bd9d2faf\") " pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.892210 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-serving-cert\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.892210 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892135 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-config-volume\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.892210 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892157 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-tmp-dir\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.892210 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892184 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpnsj\" (UniqueName: \"kubernetes.io/projected/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-kube-api-access-bpnsj\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.892412 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mr4z\" (UniqueName: \"kubernetes.io/projected/4d47a0cb-4e26-437d-9aa1-2735e9ace4f9-kube-api-access-8mr4z\") pod \"network-check-source-869dcfcdcd-d7v2f\" (UID: \"4d47a0cb-4e26-437d-9aa1-2735e9ace4f9\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" Jul 15 13:51:36.892412 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892276 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6pt\" (UniqueName: \"kubernetes.io/projected/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-kube-api-access-pp6pt\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.892412 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892318 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43e414e3-fc9e-40ee-912a-6597bd9d2faf-cert\") pod \"ingress-canary-zmd4q\" (UID: \"43e414e3-fc9e-40ee-912a-6597bd9d2faf\") " pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.892689 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892661 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-config\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.892689 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.892184 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 13:51:36.892817 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.892750 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-config-volume\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.892817 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:36.892760 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls podName:9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:37.392739471 +0000 UTC m=+35.830348260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls") pod "dns-default-dsffv" (UID: "9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5") : secret "dns-default-metrics-tls" not found Jul 15 13:51:36.893048 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.893026 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-tmp-dir\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.894823 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.894793 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-serving-cert\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.895163 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.895147 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43e414e3-fc9e-40ee-912a-6597bd9d2faf-cert\") pod \"ingress-canary-zmd4q\" (UID: \"43e414e3-fc9e-40ee-912a-6597bd9d2faf\") " pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.904552 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.904531 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlr7h\" (UniqueName: \"kubernetes.io/projected/43e414e3-fc9e-40ee-912a-6597bd9d2faf-kube-api-access-dlr7h\") pod \"ingress-canary-zmd4q\" (UID: \"43e414e3-fc9e-40ee-912a-6597bd9d2faf\") " pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:36.905829 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.905811 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpnsj\" (UniqueName: \"kubernetes.io/projected/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-kube-api-access-bpnsj\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:36.906359 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.906256 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6pt\" (UniqueName: \"kubernetes.io/projected/9cd1e854-5f7e-44d5-bc9c-91fe4c24e083-kube-api-access-pp6pt\") pod \"service-ca-operator-7695fc6c8c-dtjsl\" (UID: \"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.907036 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.907000 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mr4z\" (UniqueName: \"kubernetes.io/projected/4d47a0cb-4e26-437d-9aa1-2735e9ace4f9-kube-api-access-8mr4z\") pod \"network-check-source-869dcfcdcd-d7v2f\" (UID: \"4d47a0cb-4e26-437d-9aa1-2735e9ace4f9\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" Jul 15 13:51:36.910702 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.910680 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" Jul 15 13:51:36.929876 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.929853 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" Jul 15 13:51:36.952768 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:36.952739 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zmd4q" Jul 15 13:51:37.018594 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.018560 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" Jul 15 13:51:37.196330 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.196225 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:37.196330 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.196323 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.196369 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.196398 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196422 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196475 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196489 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls podName:9f378cf9-965b-4be4-a301-ec30d18361ce nodeName:}" failed. No retries permitted until 2026-07-15 13:51:38.196470973 +0000 UTC m=+36.634079759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls") pod "cluster-samples-operator-65db858894-8lfmx" (UID: "9f378cf9-965b-4be4-a301-ec30d18361ce") : secret "samples-operator-tls" not found Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196518 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:38.196502037 +0000 UTC m=+36.634110808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : configmap references non-existent config key: service-ca.crt Jul 15 13:51:37.196549 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196540 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls podName:516d2f7f-144e-4862-9e00-fc044804030a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:38.196529991 +0000 UTC m=+36.634138767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tmpw" (UID: "516d2f7f-144e-4862-9e00-fc044804030a") : secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:37.196873 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196564 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 13:51:37.196873 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.196597 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:38.196586603 +0000 UTC m=+36.634195373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : secret "router-metrics-certs-default" not found Jul 15 13:51:37.297592 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.297561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:37.297760 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.297714 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 13:51:37.297760 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.297736 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678f7cb99d-cd67v: secret "image-registry-tls" not found Jul 15 13:51:37.297879 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.297805 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls podName:4fde022e-b51b-49fe-a86c-1e142a744c5c nodeName:}" failed. No retries permitted until 2026-07-15 13:51:38.297782819 +0000 UTC m=+36.735391609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls") pod "image-registry-678f7cb99d-cd67v" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c") : secret "image-registry-tls" not found Jul 15 13:51:37.398286 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.398251 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:37.398571 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.398436 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 13:51:37.398571 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:37.398503 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls podName:9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:38.398487548 +0000 UTC m=+36.836096321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls") pod "dns-default-dsffv" (UID: "9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5") : secret "dns-default-metrics-tls" not found Jul 15 13:51:37.671580 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.671387 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7ddh9"] Jul 15 13:51:37.678091 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.678063 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb"] Jul 15 13:51:37.679085 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.679062 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-8pcv4"] Jul 15 13:51:37.679878 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.679862 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn"] Jul 15 13:51:37.680677 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.680658 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f"] Jul 15 13:51:37.682696 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:37.682672 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47accf41_e456_4b19_9cb2_edb9c0369ac2.slice/crio-a4b8ec167be0508fffe0be31096d64da71b45bac3c0cee25127432c6d78fe69b WatchSource:0}: Error finding container a4b8ec167be0508fffe0be31096d64da71b45bac3c0cee25127432c6d78fe69b: Status 404 returned error can't find the container with id a4b8ec167be0508fffe0be31096d64da71b45bac3c0cee25127432c6d78fe69b Jul 15 13:51:37.683347 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:37.683327 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f828c0e_e03b_4683_8405_028251958c94.slice/crio-3257c0b75d27a2c4c0ce7a1e1b773c1acdd956a37b582d72fd53f659bc625ea1 WatchSource:0}: Error finding container 3257c0b75d27a2c4c0ce7a1e1b773c1acdd956a37b582d72fd53f659bc625ea1: Status 404 returned error can't find the container with id 3257c0b75d27a2c4c0ce7a1e1b773c1acdd956a37b582d72fd53f659bc625ea1 Jul 15 13:51:37.684028 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:37.684008 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f3476a_97d2_415c_ab0a_ff9fd002ff50.slice/crio-2a82fe2c8f263b9f1ed5f82db3c6be6e4b6da4ea3236d90a933157179665c589 WatchSource:0}: Error finding container 2a82fe2c8f263b9f1ed5f82db3c6be6e4b6da4ea3236d90a933157179665c589: Status 404 returned error can't find the container with id 2a82fe2c8f263b9f1ed5f82db3c6be6e4b6da4ea3236d90a933157179665c589 Jul 15 13:51:37.685961 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.685945 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zmd4q"] Jul 15 13:51:37.687056 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:37.686930 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d47a0cb_4e26_437d_9aa1_2735e9ace4f9.slice/crio-629fda7f2e2149c2fe84b09e4da1f982e6a809a239b934162458fdd193c3dd34 WatchSource:0}: Error finding container 629fda7f2e2149c2fe84b09e4da1f982e6a809a239b934162458fdd193c3dd34: Status 404 returned error can't find the container with id 629fda7f2e2149c2fe84b09e4da1f982e6a809a239b934162458fdd193c3dd34 Jul 15 13:51:37.689106 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:37.689088 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e414e3_fc9e_40ee_912a_6597bd9d2faf.slice/crio-99e6f2779a823424c752d628a5a488aec8a0c238f4654431258c516a447d9463 WatchSource:0}: Error finding container 99e6f2779a823424c752d628a5a488aec8a0c238f4654431258c516a447d9463: Status 404 returned error can't find the container with id 99e6f2779a823424c752d628a5a488aec8a0c238f4654431258c516a447d9463 Jul 15 13:51:37.691409 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:37.691392 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl"] Jul 15 13:51:37.694093 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:51:37.694072 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd1e854_5f7e_44d5_bc9c_91fe4c24e083.slice/crio-a4ce447ce5c76115942bf222553d8a96a15594df3ac71c52e803a7e06401a78a WatchSource:0}: Error finding container a4ce447ce5c76115942bf222553d8a96a15594df3ac71c52e803a7e06401a78a: Status 404 returned error can't find the container with id a4ce447ce5c76115942bf222553d8a96a15594df3ac71c52e803a7e06401a78a Jul 15 13:51:38.002205 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.002174 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:51:38.002410 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.002213 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:51:38.002410 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.002179 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:51:38.005450 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.005417 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 13:51:38.005450 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.005416 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vzz42\"" Jul 15 13:51:38.005636 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.005457 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rdkz6\"" Jul 15 13:51:38.005636 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.005499 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.204401 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.204458 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.204490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.204507 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.204644 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:40.204625339 +0000 UTC m=+38.642234110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : configmap references non-existent config key: service-ca.crt Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.204666 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.204705 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls podName:9f378cf9-965b-4be4-a301-ec30d18361ce nodeName:}" failed. No retries permitted until 2026-07-15 13:51:40.204694081 +0000 UTC m=+38.642302862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls") pod "cluster-samples-operator-65db858894-8lfmx" (UID: "9f378cf9-965b-4be4-a301-ec30d18361ce") : secret "samples-operator-tls" not found Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.205019 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.205072 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls podName:516d2f7f-144e-4862-9e00-fc044804030a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:40.205054867 +0000 UTC m=+38.642663639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tmpw" (UID: "516d2f7f-144e-4862-9e00-fc044804030a") : secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.205140 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 13:51:38.205198 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.205172 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:40.205162688 +0000 UTC m=+38.642771461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : secret "router-metrics-certs-default" not found Jul 15 13:51:38.279692 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.279644 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" event={"ID":"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083","Type":"ContainerStarted","Data":"a4ce447ce5c76115942bf222553d8a96a15594df3ac71c52e803a7e06401a78a"} Jul 15 13:51:38.284740 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.283737 2584 generic.go:356] "Generic (PLEG): container finished" podID="01aac545-69a2-4eda-97dc-e2a200140a51" containerID="9f9566ea253e93e47ec96b4fef4fd7f1ce29b1c96af35ee6f0e129114a8e0435" exitCode=0 Jul 15 13:51:38.284740 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.283827 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerDied","Data":"9f9566ea253e93e47ec96b4fef4fd7f1ce29b1c96af35ee6f0e129114a8e0435"} Jul 15 13:51:38.287157 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.287103 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" event={"ID":"d3f3476a-97d2-415c-ab0a-ff9fd002ff50","Type":"ContainerStarted","Data":"2a82fe2c8f263b9f1ed5f82db3c6be6e4b6da4ea3236d90a933157179665c589"} Jul 15 13:51:38.289868 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.289811 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pcv4" event={"ID":"9f828c0e-e03b-4683-8405-028251958c94","Type":"ContainerStarted","Data":"3257c0b75d27a2c4c0ce7a1e1b773c1acdd956a37b582d72fd53f659bc625ea1"} Jul 15 13:51:38.291766 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.291738 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" event={"ID":"4d47a0cb-4e26-437d-9aa1-2735e9ace4f9","Type":"ContainerStarted","Data":"629fda7f2e2149c2fe84b09e4da1f982e6a809a239b934162458fdd193c3dd34"} Jul 15 13:51:38.296433 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.296353 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" event={"ID":"47accf41-e456-4b19-9cb2-edb9c0369ac2","Type":"ContainerStarted","Data":"a4b8ec167be0508fffe0be31096d64da71b45bac3c0cee25127432c6d78fe69b"} Jul 15 13:51:38.299432 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.299406 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zmd4q" event={"ID":"43e414e3-fc9e-40ee-912a-6597bd9d2faf","Type":"ContainerStarted","Data":"99e6f2779a823424c752d628a5a488aec8a0c238f4654431258c516a447d9463"} Jul 15 13:51:38.302386 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.302332 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" event={"ID":"54a8e011-19a1-4f5e-a00d-e7f937aab4d2","Type":"ContainerStarted","Data":"df7750828681576e486e7644ccc07ac953a73db4a15bf38787bd8991f253848e"} Jul 15 13:51:38.305568 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.305025 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:38.305568 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.305210 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 13:51:38.305568 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.305225 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678f7cb99d-cd67v: secret "image-registry-tls" not found Jul 15 13:51:38.305568 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.305273 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls podName:4fde022e-b51b-49fe-a86c-1e142a744c5c nodeName:}" failed. No retries permitted until 2026-07-15 13:51:40.305256231 +0000 UTC m=+38.742865005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls") pod "image-registry-678f7cb99d-cd67v" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c") : secret "image-registry-tls" not found Jul 15 13:51:38.407330 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:38.406480 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:38.407330 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.407286 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 13:51:38.407509 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:38.407368 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls podName:9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:40.407346168 +0000 UTC m=+38.844954943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls") pod "dns-default-dsffv" (UID: "9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5") : secret "dns-default-metrics-tls" not found Jul 15 13:51:39.314195 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:39.314154 2584 generic.go:356] "Generic (PLEG): container finished" podID="01aac545-69a2-4eda-97dc-e2a200140a51" containerID="41856ff88fa12218f24a9f6ce65955dcbe947a56f397a1841004b4ebf9fd01f5" exitCode=0 Jul 15 13:51:39.314733 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:39.314223 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerDied","Data":"41856ff88fa12218f24a9f6ce65955dcbe947a56f397a1841004b4ebf9fd01f5"} Jul 15 13:51:40.229990 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.229728 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:40.230176 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.230063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:40.230176 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.230097 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:40.230281 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.230246 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:40.230448 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.229901 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:40.230537 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.230523 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls podName:516d2f7f-144e-4862-9e00-fc044804030a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:44.230503928 +0000 UTC m=+42.668112701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tmpw" (UID: "516d2f7f-144e-4862-9e00-fc044804030a") : secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:40.230665 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.230638 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:44.230623601 +0000 UTC m=+42.668232389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : configmap references non-existent config key: service-ca.crt Jul 15 13:51:40.230828 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.230729 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 13:51:40.230828 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.230767 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:44.230756324 +0000 UTC m=+42.668365095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : secret "router-metrics-certs-default" not found Jul 15 13:51:40.230828 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.230421 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 13:51:40.230828 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.230799 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls podName:9f378cf9-965b-4be4-a301-ec30d18361ce nodeName:}" failed. No retries permitted until 2026-07-15 13:51:44.230791339 +0000 UTC m=+42.668400112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls") pod "cluster-samples-operator-65db858894-8lfmx" (UID: "9f378cf9-965b-4be4-a301-ec30d18361ce") : secret "samples-operator-tls" not found Jul 15 13:51:40.321666 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.321627 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26sc2" event={"ID":"01aac545-69a2-4eda-97dc-e2a200140a51","Type":"ContainerStarted","Data":"a25c6c8029ca647c2f7ce20c0e3ef1fcc0d6568fa45a4526fc932da586195a7c"} Jul 15 13:51:40.323716 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.323685 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" event={"ID":"d3f3476a-97d2-415c-ab0a-ff9fd002ff50","Type":"ContainerStarted","Data":"1d6214ca1a0b547f3192440010ab40ca1829bbb54de54f11131f894641b68a9a"} Jul 15 13:51:40.331131 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.331102 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:40.331275 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.331250 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 13:51:40.331275 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.331275 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678f7cb99d-cd67v: secret "image-registry-tls" not found Jul 15 13:51:40.331456 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.331346 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls podName:4fde022e-b51b-49fe-a86c-1e142a744c5c nodeName:}" failed. No retries permitted until 2026-07-15 13:51:44.331330479 +0000 UTC m=+42.768939254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls") pod "image-registry-678f7cb99d-cd67v" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c") : secret "image-registry-tls" not found Jul 15 13:51:40.346683 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.346571 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-26sc2" podStartSLOduration=7.83471112 podStartE2EDuration="38.346549251s" totalImagesPullingTime="30.511838131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:40.346024278 +0000 UTC m=+38.783633071" watchObservedRunningTime="2026-07-15 13:51:40.346549251 +0000 UTC m=+38.784158044" Jul 15 13:51:40.362813 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.362577 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h2twb" podStartSLOduration=6.241819942 podStartE2EDuration="8.362558541s" totalImagesPullingTime="2.120738599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:40.362430241 +0000 UTC m=+38.800039035" watchObservedRunningTime="2026-07-15 13:51:40.362558541 +0000 UTC m=+38.800167337" Jul 15 13:51:40.431840 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:40.431803 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:40.432017 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.431933 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 13:51:40.432017 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:40.431987 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls podName:9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:44.431973162 +0000 UTC m=+42.869581933 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls") pod "dns-default-dsffv" (UID: "9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5") : secret "dns-default-metrics-tls" not found Jul 15 13:51:44.268264 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:44.268226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:44.268288 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:44.268342 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:44.268361 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268406 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268463 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268482 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls podName:9f378cf9-965b-4be4-a301-ec30d18361ce nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.268460894 +0000 UTC m=+50.706069684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls") pod "cluster-samples-operator-65db858894-8lfmx" (UID: "9f378cf9-965b-4be4-a301-ec30d18361ce") : secret "samples-operator-tls" not found Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268500 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.268489735 +0000 UTC m=+50.706098508 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : configmap references non-existent config key: service-ca.crt Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268502 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268515 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls podName:516d2f7f-144e-4862-9e00-fc044804030a nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.268509064 +0000 UTC m=+50.706117834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tmpw" (UID: "516d2f7f-144e-4862-9e00-fc044804030a") : secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:44.268886 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.268563 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.268546834 +0000 UTC m=+50.706155626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : secret "router-metrics-certs-default" not found Jul 15 13:51:44.369830 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:44.369789 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:44.369998 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.369954 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 13:51:44.369998 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.369972 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678f7cb99d-cd67v: secret "image-registry-tls" not found Jul 15 13:51:44.370084 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.370024 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls podName:4fde022e-b51b-49fe-a86c-1e142a744c5c nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.370010117 +0000 UTC m=+50.807618887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls") pod "image-registry-678f7cb99d-cd67v" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c") : secret "image-registry-tls" not found Jul 15 13:51:44.471139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:44.471093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:44.471358 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.471291 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 13:51:44.471422 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:44.471382 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls podName:9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:52.471364998 +0000 UTC m=+50.908973768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls") pod "dns-default-dsffv" (UID: "9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5") : secret "dns-default-metrics-tls" not found Jul 15 13:51:46.341788 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.341756 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/0.log" Jul 15 13:51:46.342262 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.341809 2584 generic.go:356] "Generic (PLEG): container finished" podID="9f828c0e-e03b-4683-8405-028251958c94" containerID="f1fa5fbc9da5d9a7514badd96e83b984930f273d0c40f07ce102418fe9fc4abe" exitCode=255 Jul 15 13:51:46.342262 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.342075 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pcv4" event={"ID":"9f828c0e-e03b-4683-8405-028251958c94","Type":"ContainerDied","Data":"f1fa5fbc9da5d9a7514badd96e83b984930f273d0c40f07ce102418fe9fc4abe"} Jul 15 13:51:46.342262 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.342143 2584 scope.go:117] "RemoveContainer" containerID="f1fa5fbc9da5d9a7514badd96e83b984930f273d0c40f07ce102418fe9fc4abe" Jul 15 13:51:46.344111 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.344070 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" event={"ID":"4d47a0cb-4e26-437d-9aa1-2735e9ace4f9","Type":"ContainerStarted","Data":"05ad183473232bb615b1a5f615e2ee14505d6a0d0898a4c1af797c0684bedb67"} Jul 15 13:51:46.346033 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.345987 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" event={"ID":"47accf41-e456-4b19-9cb2-edb9c0369ac2","Type":"ContainerStarted","Data":"b9eb0c9fd6fdb2dac3e7f027152465198102273bd117473ef8ad4459f499d80d"} Jul 15 13:51:46.348180 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.347851 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zmd4q" event={"ID":"43e414e3-fc9e-40ee-912a-6597bd9d2faf","Type":"ContainerStarted","Data":"5a64d493526fd4c7264af857df81fd21cafc08cf848341cb0594734d580af976"} Jul 15 13:51:46.349413 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.349370 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" event={"ID":"54a8e011-19a1-4f5e-a00d-e7f937aab4d2","Type":"ContainerStarted","Data":"ec448b6613c65b47c30cd842eeb6c139c165d3c131db7f7b5e772475be4a369c"} Jul 15 13:51:46.350817 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.350795 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" event={"ID":"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083","Type":"ContainerStarted","Data":"635b0a378b37e639a318c6a715157c4da408b16123271214594e5741de38c032"} Jul 15 13:51:46.379751 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.379707 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" podStartSLOduration=6.664736442 podStartE2EDuration="14.37969233s" totalImagesPullingTime="7.714955888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:46.379502175 +0000 UTC m=+44.817110968" watchObservedRunningTime="2026-07-15 13:51:46.37969233 +0000 UTC m=+44.817301123" Jul 15 13:51:46.398355 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.398288 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" podStartSLOduration=3.697678259 podStartE2EDuration="11.39826737s" totalImagesPullingTime="7.700589111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:46.396143682 +0000 UTC m=+44.833752476" watchObservedRunningTime="2026-07-15 13:51:46.39826737 +0000 UTC m=+44.835876168" Jul 15 13:51:46.413129 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.413081 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d7v2f" podStartSLOduration=2.703816114 podStartE2EDuration="10.413064642s" totalImagesPullingTime="7.709248528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:46.412688045 +0000 UTC m=+44.850296838" watchObservedRunningTime="2026-07-15 13:51:46.413064642 +0000 UTC m=+44.850673438" Jul 15 13:51:46.437469 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.437415 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" podStartSLOduration=5.737263901 podStartE2EDuration="13.437393177s" totalImagesPullingTime="7.700129276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:46.437268542 +0000 UTC m=+44.874877335" watchObservedRunningTime="2026-07-15 13:51:46.437393177 +0000 UTC m=+44.875001973" Jul 15 13:51:46.460558 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.460497 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zmd4q" podStartSLOduration=2.760183392 podStartE2EDuration="10.46047424s" totalImagesPullingTime="7.700290848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:51:46.457883189 +0000 UTC m=+44.895491982" watchObservedRunningTime="2026-07-15 13:51:46.46047424 +0000 UTC m=+44.898083023" Jul 15 13:51:46.769523 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.769423 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:46.769523 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:46.769477 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:47.144614 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.144578 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-swh9f"] Jul 15 13:51:47.179356 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.179323 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-swh9f"] Jul 15 13:51:47.179518 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.179416 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.183218 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.183195 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 13:51:47.184087 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.184067 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8t982\"" Jul 15 13:51:47.186099 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.186079 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 13:51:47.199147 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.199124 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/facfeec1-06ad-463c-8d29-55dd1333df11-data-volume\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.199374 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.199356 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbvvm\" (UniqueName: \"kubernetes.io/projected/facfeec1-06ad-463c-8d29-55dd1333df11-kube-api-access-sbvvm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.199428 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.199389 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/facfeec1-06ad-463c-8d29-55dd1333df11-crio-socket\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.199428 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.199410 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.199559 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.199445 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.300786 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.300750 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbvvm\" (UniqueName: \"kubernetes.io/projected/facfeec1-06ad-463c-8d29-55dd1333df11-kube-api-access-sbvvm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.300978 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.300804 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/facfeec1-06ad-463c-8d29-55dd1333df11-crio-socket\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.300978 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.300834 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.300978 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.300872 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.300978 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.300908 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/facfeec1-06ad-463c-8d29-55dd1333df11-data-volume\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.304763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.301342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/facfeec1-06ad-463c-8d29-55dd1333df11-crio-socket\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.304763 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.301458 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 13:51:47.304763 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.301555 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:47.801529184 +0000 UTC m=+46.239137970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : secret "insights-runtime-extractor-tls" not found Jul 15 13:51:47.304763 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.301573 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 13:51:47.304763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.301638 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/facfeec1-06ad-463c-8d29-55dd1333df11-data-volume\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.304763 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.301774 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:47.801754932 +0000 UTC m=+46.239363716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : configmap "kube-rbac-proxy" not found Jul 15 13:51:47.321893 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.321860 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbvvm\" (UniqueName: \"kubernetes.io/projected/facfeec1-06ad-463c-8d29-55dd1333df11-kube-api-access-sbvvm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.355622 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.355592 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/1.log" Jul 15 13:51:47.356193 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.356074 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/0.log" Jul 15 13:51:47.356193 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.356111 2584 generic.go:356] "Generic (PLEG): container finished" podID="9f828c0e-e03b-4683-8405-028251958c94" containerID="f1dce80f0c5867874c168774c44e718b280f033758c1cdee96746663baa4f9e2" exitCode=255 Jul 15 13:51:47.356307 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.356231 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pcv4" event={"ID":"9f828c0e-e03b-4683-8405-028251958c94","Type":"ContainerDied","Data":"f1dce80f0c5867874c168774c44e718b280f033758c1cdee96746663baa4f9e2"} Jul 15 13:51:47.366433 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.366403 2584 scope.go:117] "RemoveContainer" containerID="f1fa5fbc9da5d9a7514badd96e83b984930f273d0c40f07ce102418fe9fc4abe" Jul 15 13:51:47.366539 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.366520 2584 scope.go:117] "RemoveContainer" containerID="f1dce80f0c5867874c168774c44e718b280f033758c1cdee96746663baa4f9e2" Jul 15 13:51:47.366761 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.366737 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8pcv4_openshift-console-operator(9f828c0e-e03b-4683-8405-028251958c94)\"" pod="openshift-console-operator/console-operator-7866497494-8pcv4" podUID="9f828c0e-e03b-4683-8405-028251958c94" Jul 15 13:51:47.806473 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.806391 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.806473 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:47.806432 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:47.806647 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.806512 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 13:51:47.806647 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.806585 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:48.806570134 +0000 UTC m=+47.244178909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : configmap "kube-rbac-proxy" not found Jul 15 13:51:47.806647 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.806610 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 13:51:47.806758 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:47.806688 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:48.806673889 +0000 UTC m=+47.244282667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : secret "insights-runtime-extractor-tls" not found Jul 15 13:51:48.361105 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:48.361080 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/1.log" Jul 15 13:51:48.361493 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:48.361425 2584 scope.go:117] "RemoveContainer" containerID="f1dce80f0c5867874c168774c44e718b280f033758c1cdee96746663baa4f9e2" Jul 15 13:51:48.361580 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:48.361562 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8pcv4_openshift-console-operator(9f828c0e-e03b-4683-8405-028251958c94)\"" pod="openshift-console-operator/console-operator-7866497494-8pcv4" podUID="9f828c0e-e03b-4683-8405-028251958c94" Jul 15 13:51:48.517010 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:48.516983 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9whzn_9618dd0e-33ad-4458-88d6-7442904afcf4/dns-node-resolver/0.log" Jul 15 13:51:48.815560 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:48.815475 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:48.815560 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:48.815519 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:48.815770 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:48.815580 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 13:51:48.815770 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:48.815631 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 13:51:48.815770 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:48.815649 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:50.815634341 +0000 UTC m=+49.253243111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : configmap "kube-rbac-proxy" not found Jul 15 13:51:48.815770 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:48.815681 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:50.81566699 +0000 UTC m=+49.253275759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : secret "insights-runtime-extractor-tls" not found Jul 15 13:51:49.516777 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:49.516752 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fl65z_4796b9f6-2024-4e6c-a008-8f3060ea1b4e/node-ca/0.log" Jul 15 13:51:50.519983 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:50.519954 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zmd4q_43e414e3-fc9e-40ee-912a-6597bd9d2faf/serve-healthcheck-canary/0.log" Jul 15 13:51:50.833654 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:50.833568 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:50.833654 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:50.833613 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:50.833832 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:50.833693 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 13:51:50.833832 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:50.833714 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 13:51:50.833832 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:50.833773 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:54.83375759 +0000 UTC m=+53.271366361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : configmap "kube-rbac-proxy" not found Jul 15 13:51:50.833832 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:50.833790 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:51:54.833783879 +0000 UTC m=+53.271392654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : secret "insights-runtime-extractor-tls" not found Jul 15 13:51:52.347008 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:52.346970 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:52.347034 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:52.347075 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:52.347101 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347126 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347180 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347184 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347191 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls podName:9f378cf9-965b-4be4-a301-ec30d18361ce nodeName:}" failed. No retries permitted until 2026-07-15 13:52:08.347173267 +0000 UTC m=+66.784782043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls") pod "cluster-samples-operator-65db858894-8lfmx" (UID: "9f378cf9-965b-4be4-a301-ec30d18361ce") : secret "samples-operator-tls" not found Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347314 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:08.347282542 +0000 UTC m=+66.784891318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : configmap references non-existent config key: service-ca.crt Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347337 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls podName:516d2f7f-144e-4862-9e00-fc044804030a nodeName:}" failed. No retries permitted until 2026-07-15 13:52:08.347327083 +0000 UTC m=+66.784935853 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tmpw" (UID: "516d2f7f-144e-4862-9e00-fc044804030a") : secret "cluster-monitoring-operator-tls" not found Jul 15 13:51:52.347450 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.347355 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs podName:864878a2-8862-420c-886d-0d7b2cf462c9 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:08.34734702 +0000 UTC m=+66.784955790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs") pod "router-default-66c5645f8-m26q2" (UID: "864878a2-8862-420c-886d-0d7b2cf462c9") : secret "router-metrics-certs-default" not found Jul 15 13:51:52.447880 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:52.447837 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:51:52.448031 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.447981 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 13:51:52.448031 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.448000 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678f7cb99d-cd67v: secret "image-registry-tls" not found Jul 15 13:51:52.448119 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.448054 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls podName:4fde022e-b51b-49fe-a86c-1e142a744c5c nodeName:}" failed. No retries permitted until 2026-07-15 13:52:08.448037519 +0000 UTC m=+66.885646289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls") pod "image-registry-678f7cb99d-cd67v" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c") : secret "image-registry-tls" not found Jul 15 13:51:52.548357 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:52.548325 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:51:52.548510 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.548469 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 13:51:52.548549 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:52.548528 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls podName:9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:08.548513317 +0000 UTC m=+66.986122086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls") pod "dns-default-dsffv" (UID: "9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5") : secret "dns-default-metrics-tls" not found Jul 15 13:51:54.870509 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:54.870478 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:54.870509 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:54.870516 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:54.870991 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:54.870605 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 13:51:54.870991 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:54.870660 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls podName:facfeec1-06ad-463c-8d29-55dd1333df11 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:02.870641303 +0000 UTC m=+61.308250073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-swh9f" (UID: "facfeec1-06ad-463c-8d29-55dd1333df11") : secret "insights-runtime-extractor-tls" not found Jul 15 13:51:54.871063 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:54.870986 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/facfeec1-06ad-463c-8d29-55dd1333df11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:51:56.770010 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:56.769971 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:56.770482 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:56.770064 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:51:56.770482 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:56.770452 2584 scope.go:117] "RemoveContainer" containerID="f1dce80f0c5867874c168774c44e718b280f033758c1cdee96746663baa4f9e2" Jul 15 13:51:57.393727 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:57.393697 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 13:51:57.394103 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:57.394089 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/1.log" Jul 15 13:51:57.394151 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:57.394121 2584 generic.go:356] "Generic (PLEG): container finished" podID="9f828c0e-e03b-4683-8405-028251958c94" containerID="7bc88dccc4752f9af7fe1f6ea0b9bcb80382ceaddf01debc924195d704378c7e" exitCode=255 Jul 15 13:51:57.394185 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:57.394150 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pcv4" event={"ID":"9f828c0e-e03b-4683-8405-028251958c94","Type":"ContainerDied","Data":"7bc88dccc4752f9af7fe1f6ea0b9bcb80382ceaddf01debc924195d704378c7e"} Jul 15 13:51:57.394217 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:57.394192 2584 scope.go:117] "RemoveContainer" containerID="f1dce80f0c5867874c168774c44e718b280f033758c1cdee96746663baa4f9e2" Jul 15 13:51:57.394480 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:57.394455 2584 scope.go:117] "RemoveContainer" containerID="7bc88dccc4752f9af7fe1f6ea0b9bcb80382ceaddf01debc924195d704378c7e" Jul 15 13:51:57.394636 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:51:57.394618 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-8pcv4_openshift-console-operator(9f828c0e-e03b-4683-8405-028251958c94)\"" pod="openshift-console-operator/console-operator-7866497494-8pcv4" podUID="9f828c0e-e03b-4683-8405-028251958c94" Jul 15 13:51:58.398789 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:51:58.398764 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 13:52:02.266892 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:02.266867 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p4t6d" Jul 15 13:52:02.937558 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:02.937519 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:52:02.939786 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:02.939757 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/facfeec1-06ad-463c-8d29-55dd1333df11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-swh9f\" (UID: \"facfeec1-06ad-463c-8d29-55dd1333df11\") " pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:52:03.092574 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:03.092550 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8t982\"" Jul 15 13:52:03.099820 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:03.099805 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-swh9f" Jul 15 13:52:03.222221 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:03.222142 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-swh9f"] Jul 15 13:52:03.225751 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:03.225722 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacfeec1_06ad_463c_8d29_55dd1333df11.slice/crio-21c439951590d10a21781768ace9dc82abf5992a38b43c1c11a7bf524426fe21 WatchSource:0}: Error finding container 21c439951590d10a21781768ace9dc82abf5992a38b43c1c11a7bf524426fe21: Status 404 returned error can't find the container with id 21c439951590d10a21781768ace9dc82abf5992a38b43c1c11a7bf524426fe21 Jul 15 13:52:03.414219 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:03.414185 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-swh9f" event={"ID":"facfeec1-06ad-463c-8d29-55dd1333df11","Type":"ContainerStarted","Data":"21c439951590d10a21781768ace9dc82abf5992a38b43c1c11a7bf524426fe21"} Jul 15 13:52:06.769864 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.769828 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:52:06.770360 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.769899 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:52:06.770360 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.770282 2584 scope.go:117] "RemoveContainer" containerID="7bc88dccc4752f9af7fe1f6ea0b9bcb80382ceaddf01debc924195d704378c7e" Jul 15 13:52:06.770835 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:52:06.770811 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-8pcv4_openshift-console-operator(9f828c0e-e03b-4683-8405-028251958c94)\"" pod="openshift-console-operator/console-operator-7866497494-8pcv4" podUID="9f828c0e-e03b-4683-8405-028251958c94" Jul 15 13:52:06.774811 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.774514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:52:06.774811 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.774754 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:52:06.776977 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.776957 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 13:52:06.776977 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.776974 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 13:52:06.787774 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.787744 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b731e8a7-a469-4d64-9ac5-14ced255483d-metrics-certs\") pod \"network-metrics-daemon-ncnc9\" (UID: \"b731e8a7-a469-4d64-9ac5-14ced255483d\") " pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:52:06.787918 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.787899 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf75f52c-b77a-4a9a-8b02-7e823c68380a-original-pull-secret\") pod \"global-pull-secret-syncer-t7tnx\" (UID: \"cf75f52c-b77a-4a9a-8b02-7e823c68380a\") " pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:52:06.826510 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.826483 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t7tnx" Jul 15 13:52:06.826843 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.826807 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rdkz6\"" Jul 15 13:52:06.833368 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.833344 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ncnc9" Jul 15 13:52:06.876024 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.876000 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:52:06.878611 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:06.878590 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqd8j\" (UniqueName: \"kubernetes.io/projected/3be71432-c1ef-419b-a4c1-645ea7cb8aa8-kube-api-access-kqd8j\") pod \"network-check-target-5qvnw\" (UID: \"3be71432-c1ef-419b-a4c1-645ea7cb8aa8\") " pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:52:07.119469 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:07.119440 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vzz42\"" Jul 15 13:52:07.126632 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:07.126598 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:52:08.389144 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.389071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:52:08.389619 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.389159 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:52:08.389619 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.389194 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:08.389619 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.389225 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:08.390087 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.390034 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/864878a2-8862-420c-886d-0d7b2cf462c9-service-ca-bundle\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:08.391991 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.391952 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/864878a2-8862-420c-886d-0d7b2cf462c9-metrics-certs\") pod \"router-default-66c5645f8-m26q2\" (UID: \"864878a2-8862-420c-886d-0d7b2cf462c9\") " pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:08.392606 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.392583 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f378cf9-965b-4be4-a301-ec30d18361ce-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8lfmx\" (UID: \"9f378cf9-965b-4be4-a301-ec30d18361ce\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:52:08.392606 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.392598 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516d2f7f-144e-4862-9e00-fc044804030a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tmpw\" (UID: \"516d2f7f-144e-4862-9e00-fc044804030a\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:52:08.489905 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.489875 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:08.492720 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.492682 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"image-registry-678f7cb99d-cd67v\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:08.515159 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.515019 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t7tnx"] Jul 15 13:52:08.517714 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:08.517682 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf75f52c_b77a_4a9a_8b02_7e823c68380a.slice/crio-6684a8a099fc120268fe4685a42b18d55762ddb6afce595371075eb25ba27f42 WatchSource:0}: Error finding container 6684a8a099fc120268fe4685a42b18d55762ddb6afce595371075eb25ba27f42: Status 404 returned error can't find the container with id 6684a8a099fc120268fe4685a42b18d55762ddb6afce595371075eb25ba27f42 Jul 15 13:52:08.560876 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.560852 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jfsg6\"" Jul 15 13:52:08.568247 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.568232 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:08.590241 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.590221 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:52:08.590733 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.590710 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-srk4z\"" Jul 15 13:52:08.592310 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.592274 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5-metrics-tls\") pod \"dns-default-dsffv\" (UID: \"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5\") " pod="openshift-dns/dns-default-dsffv" Jul 15 13:52:08.598931 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.598911 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" Jul 15 13:52:08.616640 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.616594 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-b2x5t\"" Jul 15 13:52:08.623576 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.623560 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" Jul 15 13:52:08.632611 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.632595 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kq22p\"" Jul 15 13:52:08.640586 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.640569 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:08.735820 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.735789 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5qvnw"] Jul 15 13:52:08.738349 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.738328 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ncnc9"] Jul 15 13:52:08.750428 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:08.750401 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be71432_c1ef_419b_a4c1_645ea7cb8aa8.slice/crio-744089a86dd4c6a28bdc8bc676d444628cc5f5cc2e0ca73210a012ff29165493 WatchSource:0}: Error finding container 744089a86dd4c6a28bdc8bc676d444628cc5f5cc2e0ca73210a012ff29165493: Status 404 returned error can't find the container with id 744089a86dd4c6a28bdc8bc676d444628cc5f5cc2e0ca73210a012ff29165493 Jul 15 13:52:08.750977 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:08.750959 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb731e8a7_a469_4d64_9ac5_14ced255483d.slice/crio-883576d718c1da05b35eca3dc2f517c001a1828b9ef61e1962cad60ff6ffc675 WatchSource:0}: Error finding container 883576d718c1da05b35eca3dc2f517c001a1828b9ef61e1962cad60ff6ffc675: Status 404 returned error can't find the container with id 883576d718c1da05b35eca3dc2f517c001a1828b9ef61e1962cad60ff6ffc675 Jul 15 13:52:08.778430 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.778410 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2df9f\"" Jul 15 13:52:08.785868 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.785847 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dsffv" Jul 15 13:52:08.965310 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.965255 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-678f7cb99d-cd67v"] Jul 15 13:52:08.968228 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:08.968198 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fde022e_b51b_49fe_a86c_1e142a744c5c.slice/crio-66d9ac0cda175184cc0bdbc8182c167914bc84dd75868512f770ede6367056e7 WatchSource:0}: Error finding container 66d9ac0cda175184cc0bdbc8182c167914bc84dd75868512f770ede6367056e7: Status 404 returned error can't find the container with id 66d9ac0cda175184cc0bdbc8182c167914bc84dd75868512f770ede6367056e7 Jul 15 13:52:08.981054 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:08.980049 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw"] Jul 15 13:52:08.986309 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:08.986273 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516d2f7f_144e_4862_9e00_fc044804030a.slice/crio-9ba7d0d141cd90571e3220a4bf3591762e651a8ce3bb37f5d773f30da4c6f390 WatchSource:0}: Error finding container 9ba7d0d141cd90571e3220a4bf3591762e651a8ce3bb37f5d773f30da4c6f390: Status 404 returned error can't find the container with id 9ba7d0d141cd90571e3220a4bf3591762e651a8ce3bb37f5d773f30da4c6f390 Jul 15 13:52:09.035124 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.035098 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dsffv"] Jul 15 13:52:09.038786 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:09.038750 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5b5684_9f66_4e5a_ac0c_94b9f1f4a8b5.slice/crio-e540623bdad9ac625ab49b888292bc93f81b6a48a1b428da0f73edf80e7d1f0c WatchSource:0}: Error finding container e540623bdad9ac625ab49b888292bc93f81b6a48a1b428da0f73edf80e7d1f0c: Status 404 returned error can't find the container with id e540623bdad9ac625ab49b888292bc93f81b6a48a1b428da0f73edf80e7d1f0c Jul 15 13:52:09.204679 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.204557 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx"] Jul 15 13:52:09.209485 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.209413 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66c5645f8-m26q2"] Jul 15 13:52:09.213716 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:09.213692 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864878a2_8862_420c_886d_0d7b2cf462c9.slice/crio-4bf5852ec9c4d218a6b15b558ea8bfa9ae4a1e0cb134ce496d8b4eede898e966 WatchSource:0}: Error finding container 4bf5852ec9c4d218a6b15b558ea8bfa9ae4a1e0cb134ce496d8b4eede898e966: Status 404 returned error can't find the container with id 4bf5852ec9c4d218a6b15b558ea8bfa9ae4a1e0cb134ce496d8b4eede898e966 Jul 15 13:52:09.442073 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.441648 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66c5645f8-m26q2" event={"ID":"864878a2-8862-420c-886d-0d7b2cf462c9","Type":"ContainerStarted","Data":"665b1adbb6ea1c248f4fefb103f855c87ed139eaea1677d9fbb73f641a28e8f5"} Jul 15 13:52:09.442073 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.441720 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66c5645f8-m26q2" event={"ID":"864878a2-8862-420c-886d-0d7b2cf462c9","Type":"ContainerStarted","Data":"4bf5852ec9c4d218a6b15b558ea8bfa9ae4a1e0cb134ce496d8b4eede898e966"} Jul 15 13:52:09.443686 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.443615 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t7tnx" event={"ID":"cf75f52c-b77a-4a9a-8b02-7e823c68380a","Type":"ContainerStarted","Data":"6684a8a099fc120268fe4685a42b18d55762ddb6afce595371075eb25ba27f42"} Jul 15 13:52:09.447806 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.447775 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" event={"ID":"516d2f7f-144e-4862-9e00-fc044804030a","Type":"ContainerStarted","Data":"9ba7d0d141cd90571e3220a4bf3591762e651a8ce3bb37f5d773f30da4c6f390"} Jul 15 13:52:09.450374 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.450346 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" event={"ID":"4fde022e-b51b-49fe-a86c-1e142a744c5c","Type":"ContainerStarted","Data":"143c6ffb1dc94788a30fb605d187cfa1881486170719ff16bfd8c5c85a2e0d8e"} Jul 15 13:52:09.450480 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.450392 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" event={"ID":"4fde022e-b51b-49fe-a86c-1e142a744c5c","Type":"ContainerStarted","Data":"66d9ac0cda175184cc0bdbc8182c167914bc84dd75868512f770ede6367056e7"} Jul 15 13:52:09.450956 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.450930 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:09.454755 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.454694 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dsffv" event={"ID":"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5","Type":"ContainerStarted","Data":"e540623bdad9ac625ab49b888292bc93f81b6a48a1b428da0f73edf80e7d1f0c"} Jul 15 13:52:09.457069 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.456978 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" event={"ID":"9f378cf9-965b-4be4-a301-ec30d18361ce","Type":"ContainerStarted","Data":"b84c37f0ba11b1ff371615a44f62c3087b7a2c69a9040c812cc13d69c2a1d1ec"} Jul 15 13:52:09.458435 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.458404 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ncnc9" event={"ID":"b731e8a7-a469-4d64-9ac5-14ced255483d","Type":"ContainerStarted","Data":"883576d718c1da05b35eca3dc2f517c001a1828b9ef61e1962cad60ff6ffc675"} Jul 15 13:52:09.460713 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.460641 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-swh9f" event={"ID":"facfeec1-06ad-463c-8d29-55dd1333df11","Type":"ContainerStarted","Data":"eec9f7d9dc464a43051844b79cb718505c66c365ba7952b15bca37553183361d"} Jul 15 13:52:09.463935 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.463373 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5qvnw" event={"ID":"3be71432-c1ef-419b-a4c1-645ea7cb8aa8","Type":"ContainerStarted","Data":"f1f4af7fdd2e148b5769931d7c2d50ffb8c0aa36c4881f2ab4eef92348a2fed4"} Jul 15 13:52:09.463935 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.463439 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5qvnw" event={"ID":"3be71432-c1ef-419b-a4c1-645ea7cb8aa8","Type":"ContainerStarted","Data":"744089a86dd4c6a28bdc8bc676d444628cc5f5cc2e0ca73210a012ff29165493"} Jul 15 13:52:09.463935 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.463884 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:52:09.464329 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.464269 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-66c5645f8-m26q2" podStartSLOduration=37.464251039 podStartE2EDuration="37.464251039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:09.462500935 +0000 UTC m=+67.900109744" watchObservedRunningTime="2026-07-15 13:52:09.464251039 +0000 UTC m=+67.901859843" Jul 15 13:52:09.485251 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.484918 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" podStartSLOduration=37.484899942 podStartE2EDuration="37.484899942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:09.483634358 +0000 UTC m=+67.921243151" watchObservedRunningTime="2026-07-15 13:52:09.484899942 +0000 UTC m=+67.922508736" Jul 15 13:52:09.500798 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.500675 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5qvnw" podStartSLOduration=67.500657389 podStartE2EDuration="1m7.500657389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:09.49996729 +0000 UTC m=+67.937576086" watchObservedRunningTime="2026-07-15 13:52:09.500657389 +0000 UTC m=+67.938266162" Jul 15 13:52:09.570514 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.569106 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:09.572538 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:09.572515 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:10.474928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:10.474898 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ncnc9" event={"ID":"b731e8a7-a469-4d64-9ac5-14ced255483d","Type":"ContainerStarted","Data":"3420f7c6cbe89e956618cf934e59dd842c48e29012f4e8a22cb1cf71b554fd11"} Jul 15 13:52:10.476663 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:10.475411 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:10.476783 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:10.476725 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-66c5645f8-m26q2" Jul 15 13:52:11.390520 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.390476 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn"] Jul 15 13:52:11.394679 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.394648 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt"] Jul 15 13:52:11.394819 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.394803 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.399533 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.399374 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 13:52:11.401282 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.400338 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 15 13:52:11.401282 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.400670 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vhkb4\"" Jul 15 13:52:11.401282 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.400896 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 13:52:11.401282 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.401132 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 13:52:11.411657 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.411580 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.415453 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.415404 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn"] Jul 15 13:52:11.415559 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.415528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 15 13:52:11.416412 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.416392 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt"] Jul 15 13:52:11.421605 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.421550 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-678f7cb99d-cd67v"] Jul 15 13:52:11.486069 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.486038 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ncnc9" event={"ID":"b731e8a7-a469-4d64-9ac5-14ced255483d","Type":"ContainerStarted","Data":"4f2b68e0042b07ce086de3428d230f4df36799f31be107cb0a33684003d03fcb"} Jul 15 13:52:11.508148 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.507005 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74ddfb6f4d-j8lvr"] Jul 15 13:52:11.510834 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.510812 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.523911 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.523824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl652\" (UniqueName: \"kubernetes.io/projected/68e74ded-4218-4f55-b5d7-f38d972540ad-kube-api-access-zl652\") pod \"managed-serviceaccount-addon-agent-6444b9b95d-26wtn\" (UID: \"68e74ded-4218-4f55-b5d7-f38d972540ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.523911 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.523861 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c28b80c4-1901-4164-98f5-6e56652f297c-tmp\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.524082 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.523949 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68e74ded-4218-4f55-b5d7-f38d972540ad-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6444b9b95d-26wtn\" (UID: \"68e74ded-4218-4f55-b5d7-f38d972540ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.524082 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.523982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97kwl\" (UniqueName: \"kubernetes.io/projected/c28b80c4-1901-4164-98f5-6e56652f297c-kube-api-access-97kwl\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.524082 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.524042 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c28b80c4-1901-4164-98f5-6e56652f297c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.529816 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.529765 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ncnc9" podStartSLOduration=68.036263555 podStartE2EDuration="1m9.529748418s" totalImagesPullingTime="1.493484863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:11.516889642 +0000 UTC m=+69.954498435" watchObservedRunningTime="2026-07-15 13:52:11.529748418 +0000 UTC m=+69.967357211" Jul 15 13:52:11.530787 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.530762 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74ddfb6f4d-j8lvr"] Jul 15 13:52:11.624651 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624620 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-registry-tls\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.624824 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624660 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7177928d-8085-4aea-b14a-bb17c4dca4d9-registry-certificates\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.624824 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624693 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl652\" (UniqueName: \"kubernetes.io/projected/68e74ded-4218-4f55-b5d7-f38d972540ad-kube-api-access-zl652\") pod \"managed-serviceaccount-addon-agent-6444b9b95d-26wtn\" (UID: \"68e74ded-4218-4f55-b5d7-f38d972540ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.624824 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624722 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7177928d-8085-4aea-b14a-bb17c4dca4d9-ca-trust-extracted\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.624824 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624794 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c28b80c4-1901-4164-98f5-6e56652f297c-tmp\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.625058 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624878 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7177928d-8085-4aea-b14a-bb17c4dca4d9-trusted-ca\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.625058 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.624966 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68e74ded-4218-4f55-b5d7-f38d972540ad-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6444b9b95d-26wtn\" (UID: \"68e74ded-4218-4f55-b5d7-f38d972540ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.625058 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625033 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7177928d-8085-4aea-b14a-bb17c4dca4d9-image-registry-private-configuration\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.625227 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625066 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97kwl\" (UniqueName: \"kubernetes.io/projected/c28b80c4-1901-4164-98f5-6e56652f297c-kube-api-access-97kwl\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.625227 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625171 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7177928d-8085-4aea-b14a-bb17c4dca4d9-installation-pull-secrets\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.625227 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625198 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phlvh\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-kube-api-access-phlvh\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.625399 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625242 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c28b80c4-1901-4164-98f5-6e56652f297c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.625399 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625283 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-bound-sa-token\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.625688 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.625539 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c28b80c4-1901-4164-98f5-6e56652f297c-tmp\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.628876 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.628849 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68e74ded-4218-4f55-b5d7-f38d972540ad-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6444b9b95d-26wtn\" (UID: \"68e74ded-4218-4f55-b5d7-f38d972540ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.629073 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.629050 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c28b80c4-1901-4164-98f5-6e56652f297c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.634494 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.634469 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97kwl\" (UniqueName: \"kubernetes.io/projected/c28b80c4-1901-4164-98f5-6e56652f297c-kube-api-access-97kwl\") pod \"klusterlet-addon-workmgr-6599f86c84-htqxt\" (UID: \"c28b80c4-1901-4164-98f5-6e56652f297c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.635200 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.635175 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl652\" (UniqueName: \"kubernetes.io/projected/68e74ded-4218-4f55-b5d7-f38d972540ad-kube-api-access-zl652\") pod \"managed-serviceaccount-addon-agent-6444b9b95d-26wtn\" (UID: \"68e74ded-4218-4f55-b5d7-f38d972540ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.716136 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.716062 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" Jul 15 13:52:11.726622 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726592 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7177928d-8085-4aea-b14a-bb17c4dca4d9-trusted-ca\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.726763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7177928d-8085-4aea-b14a-bb17c4dca4d9-image-registry-private-configuration\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.726851 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7177928d-8085-4aea-b14a-bb17c4dca4d9-installation-pull-secrets\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.726905 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726873 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phlvh\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-kube-api-access-phlvh\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.726947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726908 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-bound-sa-token\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.727002 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726957 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-registry-tls\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.727002 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.726984 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7177928d-8085-4aea-b14a-bb17c4dca4d9-registry-certificates\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.727094 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.727016 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7177928d-8085-4aea-b14a-bb17c4dca4d9-ca-trust-extracted\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.727517 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.727387 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7177928d-8085-4aea-b14a-bb17c4dca4d9-ca-trust-extracted\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.727648 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.727627 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7177928d-8085-4aea-b14a-bb17c4dca4d9-trusted-ca\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.728116 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.728072 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7177928d-8085-4aea-b14a-bb17c4dca4d9-registry-certificates\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.729221 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.729132 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:11.729996 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.729673 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7177928d-8085-4aea-b14a-bb17c4dca4d9-installation-pull-secrets\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.730102 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.730049 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7177928d-8085-4aea-b14a-bb17c4dca4d9-image-registry-private-configuration\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.731307 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.731271 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-registry-tls\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.736339 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.736319 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-bound-sa-token\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.736669 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.736646 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phlvh\" (UniqueName: \"kubernetes.io/projected/7177928d-8085-4aea-b14a-bb17c4dca4d9-kube-api-access-phlvh\") pod \"image-registry-74ddfb6f4d-j8lvr\" (UID: \"7177928d-8085-4aea-b14a-bb17c4dca4d9\") " pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:11.822149 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:11.822100 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:16.446458 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:16.446383 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt"] Jul 15 13:52:16.512602 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:16.512564 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28b80c4_1901_4164_98f5_6e56652f297c.slice/crio-644f8faed8969e3291659bf428effd68c98e83f9c27a6de72be5736722c83960 WatchSource:0}: Error finding container 644f8faed8969e3291659bf428effd68c98e83f9c27a6de72be5736722c83960: Status 404 returned error can't find the container with id 644f8faed8969e3291659bf428effd68c98e83f9c27a6de72be5736722c83960 Jul 15 13:52:16.658008 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:16.657973 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74ddfb6f4d-j8lvr"] Jul 15 13:52:16.663350 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:16.663320 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7177928d_8085_4aea_b14a_bb17c4dca4d9.slice/crio-6b805d53dbf48dcf46510cf1a9a0f4a21d1435b67f54f272128d5fe20989aa6a WatchSource:0}: Error finding container 6b805d53dbf48dcf46510cf1a9a0f4a21d1435b67f54f272128d5fe20989aa6a: Status 404 returned error can't find the container with id 6b805d53dbf48dcf46510cf1a9a0f4a21d1435b67f54f272128d5fe20989aa6a Jul 15 13:52:16.675839 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:16.675801 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn"] Jul 15 13:52:16.678771 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:16.678720 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68e74ded_4218_4f55_b5d7_f38d972540ad.slice/crio-2095432a630cd7993356a8c9d0ab73efcd17cfc9c882a62af4e0d3a740658364 WatchSource:0}: Error finding container 2095432a630cd7993356a8c9d0ab73efcd17cfc9c882a62af4e0d3a740658364: Status 404 returned error can't find the container with id 2095432a630cd7993356a8c9d0ab73efcd17cfc9c882a62af4e0d3a740658364 Jul 15 13:52:17.524786 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.524750 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t7tnx" event={"ID":"cf75f52c-b77a-4a9a-8b02-7e823c68380a","Type":"ContainerStarted","Data":"b886450155397537bdd9e6515858b8a18196b39671d8a9214caaebd963237535"} Jul 15 13:52:17.530792 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.530737 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" event={"ID":"516d2f7f-144e-4862-9e00-fc044804030a","Type":"ContainerStarted","Data":"46d7ee8e9c40e77bf69f2e891b75c74eefa7e467fb00a5bd9c807bc648ca07e5"} Jul 15 13:52:17.534340 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.534012 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" event={"ID":"7177928d-8085-4aea-b14a-bb17c4dca4d9","Type":"ContainerStarted","Data":"98ab33cecc2890b566c7b84cf2886d168e43e55e8403cf268cd56da475f01c89"} Jul 15 13:52:17.534340 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.534058 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" event={"ID":"7177928d-8085-4aea-b14a-bb17c4dca4d9","Type":"ContainerStarted","Data":"6b805d53dbf48dcf46510cf1a9a0f4a21d1435b67f54f272128d5fe20989aa6a"} Jul 15 13:52:17.534609 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.534563 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:17.538423 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.537914 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dsffv" event={"ID":"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5","Type":"ContainerStarted","Data":"ea265b5d5a3e197b930bbfcdd386cde112beed61d00a0ffc26a60e7de911e578"} Jul 15 13:52:17.538423 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.537957 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dsffv" event={"ID":"9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5","Type":"ContainerStarted","Data":"c32bd1b95b0637e98766765144c4a1dac7538f81a386db590b53075b79ca6deb"} Jul 15 13:52:17.538423 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.537990 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dsffv" Jul 15 13:52:17.540729 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.540702 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" event={"ID":"9f378cf9-965b-4be4-a301-ec30d18361ce","Type":"ContainerStarted","Data":"64f085873afb66b32f49f3869ebb6406db2f77094aedd6efab91df237b2b5f36"} Jul 15 13:52:17.540834 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.540755 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" event={"ID":"9f378cf9-965b-4be4-a301-ec30d18361ce","Type":"ContainerStarted","Data":"7ca61d4da6ff773ed531c3679bd1e8b4880b9f7667300e00086417cbaea23051"} Jul 15 13:52:17.543154 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.542746 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" event={"ID":"68e74ded-4218-4f55-b5d7-f38d972540ad","Type":"ContainerStarted","Data":"2095432a630cd7993356a8c9d0ab73efcd17cfc9c882a62af4e0d3a740658364"} Jul 15 13:52:17.546796 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.546773 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-swh9f" event={"ID":"facfeec1-06ad-463c-8d29-55dd1333df11","Type":"ContainerStarted","Data":"6e9133310c4e02fb2a6f2c7c5d45b3adaa0bf8aeff62d35d67b6f838590a5262"} Jul 15 13:52:17.548492 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.548461 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" event={"ID":"c28b80c4-1901-4164-98f5-6e56652f297c","Type":"ContainerStarted","Data":"644f8faed8969e3291659bf428effd68c98e83f9c27a6de72be5736722c83960"} Jul 15 13:52:17.572321 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.571844 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t7tnx" podStartSLOduration=67.578316768 podStartE2EDuration="1m15.571825213s" totalImagesPullingTime="7.993508445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:17.544494507 +0000 UTC m=+75.982103301" watchObservedRunningTime="2026-07-15 13:52:17.571825213 +0000 UTC m=+76.009434004" Jul 15 13:52:17.572321 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.572162 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" podStartSLOduration=6.572152454 podStartE2EDuration="6.572152454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:17.570744154 +0000 UTC m=+76.008352948" watchObservedRunningTime="2026-07-15 13:52:17.572152454 +0000 UTC m=+76.009761248" Jul 15 13:52:17.592827 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.592679 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dsffv" podStartSLOduration=34.123981064 podStartE2EDuration="41.592668412s" totalImagesPullingTime="7.468687348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:17.591234208 +0000 UTC m=+76.028843007" watchObservedRunningTime="2026-07-15 13:52:17.592668412 +0000 UTC m=+76.030277208" Jul 15 13:52:17.612642 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.611682 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tmpw" podStartSLOduration=39.173003562 podStartE2EDuration="45.611665124s" totalImagesPullingTime="6.438661562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:17.611027559 +0000 UTC m=+76.048636353" watchObservedRunningTime="2026-07-15 13:52:17.611665124 +0000 UTC m=+76.049273917" Jul 15 13:52:17.638314 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:17.637749 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8lfmx" podStartSLOduration=38.745508134 podStartE2EDuration="45.637731685s" totalImagesPullingTime="6.892223551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:17.637505053 +0000 UTC m=+76.075113846" watchObservedRunningTime="2026-07-15 13:52:17.637731685 +0000 UTC m=+76.075340479" Jul 15 13:52:22.004705 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:22.004673 2584 scope.go:117] "RemoveContainer" containerID="7bc88dccc4752f9af7fe1f6ea0b9bcb80382ceaddf01debc924195d704378c7e" Jul 15 13:52:23.576446 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.576413 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 13:52:23.576899 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.576537 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pcv4" event={"ID":"9f828c0e-e03b-4683-8405-028251958c94","Type":"ContainerStarted","Data":"daa6ce4ad8e38deb5cc8aa499218f6c6695ab367bed89b2c3e1b7357e4db6830"} Jul 15 13:52:23.576899 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.576856 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:52:23.578383 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.578335 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" event={"ID":"68e74ded-4218-4f55-b5d7-f38d972540ad","Type":"ContainerStarted","Data":"5930596d75e97e030a4e3c05ccb55947dec889a55a410c2a61eb43b20f94698f"} Jul 15 13:52:23.584837 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.584807 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-8pcv4" Jul 15 13:52:23.585874 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.585842 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-swh9f" event={"ID":"facfeec1-06ad-463c-8d29-55dd1333df11","Type":"ContainerStarted","Data":"0f803ef9b3e2f74a0716c9f3bf8341f7e45fec5b8ceb9bca6eb99f2764be75eb"} Jul 15 13:52:23.587438 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.587415 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" event={"ID":"c28b80c4-1901-4164-98f5-6e56652f297c","Type":"ContainerStarted","Data":"9df04236b0d815b271820ac413d942ff74efb4be4a527d8d484e67e815cb24c6"} Jul 15 13:52:23.587609 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.587594 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:23.589058 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.589032 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" Jul 15 13:52:23.602355 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.602286 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-8pcv4" podStartSLOduration=43.902653976 podStartE2EDuration="51.602267686s" totalImagesPullingTime="7.69961371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:23.601396598 +0000 UTC m=+82.039005393" watchObservedRunningTime="2026-07-15 13:52:23.602267686 +0000 UTC m=+82.039876478" Jul 15 13:52:23.626476 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.626429 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-swh9f" podStartSLOduration=17.183017593 podStartE2EDuration="36.626413776s" totalImagesPullingTime="19.443396183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:51:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:23.62495838 +0000 UTC m=+82.062567174" watchObservedRunningTime="2026-07-15 13:52:23.626413776 +0000 UTC m=+82.064022569" Jul 15 13:52:23.647351 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.647288 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6444b9b95d-26wtn" podStartSLOduration=6.383611539 podStartE2EDuration="12.647274589s" totalImagesPullingTime="6.26366305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:23.646179134 +0000 UTC m=+82.083787927" watchObservedRunningTime="2026-07-15 13:52:23.647274589 +0000 UTC m=+82.084883381" Jul 15 13:52:23.669032 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:23.668995 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6599f86c84-htqxt" podStartSLOduration=6.218039976 podStartE2EDuration="12.668980289s" totalImagesPullingTime="6.450940313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:23.668632018 +0000 UTC m=+82.106240810" watchObservedRunningTime="2026-07-15 13:52:23.668980289 +0000 UTC m=+82.106589080" Jul 15 13:52:26.934653 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.934544 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q9php"] Jul 15 13:52:26.939273 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.939250 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.942700 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.942677 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 13:52:26.942894 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.942873 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 13:52:26.943010 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.942946 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 13:52:26.943742 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.943726 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-64bzp\"" Jul 15 13:52:26.943853 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.943729 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 13:52:26.951820 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.951798 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-tls\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.951915 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.951835 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-sys\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.951915 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.951884 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnx2k\" (UniqueName: \"kubernetes.io/projected/02f12689-395c-444a-996e-e539ca2ab968-kube-api-access-lnx2k\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.951991 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.951946 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-wtmp\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.951991 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.951969 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02f12689-395c-444a-996e-e539ca2ab968-metrics-client-ca\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.951991 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.951988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-accelerators-collector-config\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.952101 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.952010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.952101 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.952030 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-root\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:26.952101 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:26.952059 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-textfile\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.052870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.052833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-wtmp\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.052870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.052872 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02f12689-395c-444a-996e-e539ca2ab968-metrics-client-ca\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.052889 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-accelerators-collector-config\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.052915 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.052943 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-root\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.052995 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-root\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053030 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-wtmp\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053048 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-textfile\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053128 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053125 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-tls\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053541 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053157 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-sys\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053541 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053222 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnx2k\" (UniqueName: \"kubernetes.io/projected/02f12689-395c-444a-996e-e539ca2ab968-kube-api-access-lnx2k\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053541 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:52:27.053257 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 13:52:27.053541 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053316 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02f12689-395c-444a-996e-e539ca2ab968-sys\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053541 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:52:27.053356 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-tls podName:02f12689-395c-444a-996e-e539ca2ab968 nodeName:}" failed. No retries permitted until 2026-07-15 13:52:27.553332127 +0000 UTC m=+85.990940903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-tls") pod "node-exporter-q9php" (UID: "02f12689-395c-444a-996e-e539ca2ab968") : secret "node-exporter-tls" not found Jul 15 13:52:27.053783 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053622 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-textfile\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053783 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053702 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02f12689-395c-444a-996e-e539ca2ab968-metrics-client-ca\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.053783 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.053733 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-accelerators-collector-config\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.055630 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.055609 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.066606 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.066587 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnx2k\" (UniqueName: \"kubernetes.io/projected/02f12689-395c-444a-996e-e539ca2ab968-kube-api-access-lnx2k\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.555765 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.555730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-tls\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.557141 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.557111 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dsffv" Jul 15 13:52:27.558116 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.558083 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02f12689-395c-444a-996e-e539ca2ab968-node-exporter-tls\") pod \"node-exporter-q9php\" (UID: \"02f12689-395c-444a-996e-e539ca2ab968\") " pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.848625 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:27.848572 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q9php" Jul 15 13:52:27.860469 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:27.860443 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f12689_395c_444a_996e_e539ca2ab968.slice/crio-7e025cb7cb601fe871ace026b3e370ffb1e2cd180ad6e940b749228fc8c92204 WatchSource:0}: Error finding container 7e025cb7cb601fe871ace026b3e370ffb1e2cd180ad6e940b749228fc8c92204: Status 404 returned error can't find the container with id 7e025cb7cb601fe871ace026b3e370ffb1e2cd180ad6e940b749228fc8c92204 Jul 15 13:52:28.614529 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:28.614493 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9php" event={"ID":"02f12689-395c-444a-996e-e539ca2ab968","Type":"ContainerStarted","Data":"7e025cb7cb601fe871ace026b3e370ffb1e2cd180ad6e940b749228fc8c92204"} Jul 15 13:52:29.619866 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:29.619831 2584 generic.go:356] "Generic (PLEG): container finished" podID="02f12689-395c-444a-996e-e539ca2ab968" containerID="a1b9e5ffde8154bcd9f6dd40cb19335149fec1c115a3b575e42c049b220d7067" exitCode=0 Jul 15 13:52:29.620287 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:29.619870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9php" event={"ID":"02f12689-395c-444a-996e-e539ca2ab968","Type":"ContainerDied","Data":"a1b9e5ffde8154bcd9f6dd40cb19335149fec1c115a3b575e42c049b220d7067"} Jul 15 13:52:30.625508 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:30.625476 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9php" event={"ID":"02f12689-395c-444a-996e-e539ca2ab968","Type":"ContainerStarted","Data":"4eb3ffda84abaefd1f9a3a92149ceda2f1e8b0fa87732f39819744947d9cd701"} Jul 15 13:52:30.625870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:30.625532 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9php" event={"ID":"02f12689-395c-444a-996e-e539ca2ab968","Type":"ContainerStarted","Data":"8edf71caf441e4db787e614470bc94bdb142fb400f3711d0fb990db635f663ba"} Jul 15 13:52:30.649729 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:30.649677 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q9php" podStartSLOduration=3.713019379 podStartE2EDuration="4.649658699s" totalImagesPullingTime="870.528551ms" totalInitContainerRuntime="66.110769ms" isStatefulPod=true podCreationTimestamp="2026-07-15 13:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:30.648167147 +0000 UTC m=+89.085775940" watchObservedRunningTime="2026-07-15 13:52:30.649658699 +0000 UTC m=+89.087267491" Jul 15 13:52:32.496081 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:32.496051 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:37.516424 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.516359 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" podUID="4fde022e-b51b-49fe-a86c-1e142a744c5c" containerName="registry" containerID="cri-o://143c6ffb1dc94788a30fb605d187cfa1881486170719ff16bfd8c5c85a2e0d8e" gracePeriod=30 Jul 15 13:52:37.654685 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.654649 2584 generic.go:356] "Generic (PLEG): container finished" podID="4fde022e-b51b-49fe-a86c-1e142a744c5c" containerID="143c6ffb1dc94788a30fb605d187cfa1881486170719ff16bfd8c5c85a2e0d8e" exitCode=0 Jul 15 13:52:37.654844 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.654732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" event={"ID":"4fde022e-b51b-49fe-a86c-1e142a744c5c","Type":"ContainerDied","Data":"143c6ffb1dc94788a30fb605d187cfa1881486170719ff16bfd8c5c85a2e0d8e"} Jul 15 13:52:37.759455 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.759432 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:37.842164 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842136 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-certificates\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842335 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842182 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842335 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842222 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4fde022e-b51b-49fe-a86c-1e142a744c5c-ca-trust-extracted\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842422 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg79g\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-kube-api-access-pg79g\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842503 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842485 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-image-registry-private-configuration\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842547 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842521 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-trusted-ca\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842597 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842572 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-installation-pull-secrets\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842647 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842605 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:52:37.842647 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842615 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-bound-sa-token\") pod \"4fde022e-b51b-49fe-a86c-1e142a744c5c\" (UID: \"4fde022e-b51b-49fe-a86c-1e142a744c5c\") " Jul 15 13:52:37.842927 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.842902 2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-certificates\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.843443 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.843385 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:52:37.845029 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.844998 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-kube-api-access-pg79g" (OuterVolumeSpecName: "kube-api-access-pg79g") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "kube-api-access-pg79g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:52:37.845144 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.845071 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:52:37.845144 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.845081 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:52:37.845261 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.845221 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:52:37.845468 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.845449 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:52:37.850762 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.850740 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fde022e-b51b-49fe-a86c-1e142a744c5c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4fde022e-b51b-49fe-a86c-1e142a744c5c" (UID: "4fde022e-b51b-49fe-a86c-1e142a744c5c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:52:37.944139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944109 2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4fde022e-b51b-49fe-a86c-1e142a744c5c-ca-trust-extracted\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.944139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944139 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg79g\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-kube-api-access-pg79g\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.944278 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944151 2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-image-registry-private-configuration\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.944278 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944161 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fde022e-b51b-49fe-a86c-1e142a744c5c-trusted-ca\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.944278 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944169 2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4fde022e-b51b-49fe-a86c-1e142a744c5c-installation-pull-secrets\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.944278 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944177 2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-bound-sa-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:37.944278 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:37.944185 2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4fde022e-b51b-49fe-a86c-1e142a744c5c-registry-tls\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:52:38.660112 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:38.660072 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" Jul 15 13:52:38.660624 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:38.660079 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678f7cb99d-cd67v" event={"ID":"4fde022e-b51b-49fe-a86c-1e142a744c5c","Type":"ContainerDied","Data":"66d9ac0cda175184cc0bdbc8182c167914bc84dd75868512f770ede6367056e7"} Jul 15 13:52:38.660624 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:38.660212 2584 scope.go:117] "RemoveContainer" containerID="143c6ffb1dc94788a30fb605d187cfa1881486170719ff16bfd8c5c85a2e0d8e" Jul 15 13:52:38.678698 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:38.678668 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-678f7cb99d-cd67v"] Jul 15 13:52:38.682930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:38.682905 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-678f7cb99d-cd67v"] Jul 15 13:52:39.562401 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:39.562371 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74ddfb6f4d-j8lvr" Jul 15 13:52:40.005905 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:40.005869 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fde022e-b51b-49fe-a86c-1e142a744c5c" path="/var/lib/kubelet/pods/4fde022e-b51b-49fe-a86c-1e142a744c5c/volumes" Jul 15 13:52:41.488555 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:41.488523 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5qvnw" Jul 15 13:52:43.699710 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.699673 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75b5597c6d-w7c9v"] Jul 15 13:52:43.700061 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.699988 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fde022e-b51b-49fe-a86c-1e142a744c5c" containerName="registry" Jul 15 13:52:43.700061 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.700002 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fde022e-b51b-49fe-a86c-1e142a744c5c" containerName="registry" Jul 15 13:52:43.700061 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.700053 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fde022e-b51b-49fe-a86c-1e142a744c5c" containerName="registry" Jul 15 13:52:43.763784 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.763753 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b5597c6d-w7c9v"] Jul 15 13:52:43.763928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.763868 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.768567 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.768540 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 13:52:43.769688 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.769668 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 13:52:43.769952 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.769934 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 13:52:43.771082 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.771065 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 13:52:43.771414 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.771382 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q8l6d\"" Jul 15 13:52:43.771414 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.771406 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 13:52:43.771586 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.771446 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 13:52:43.771586 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.771521 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 13:52:43.780757 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.780739 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 13:52:43.891152 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg5f6\" (UniqueName: \"kubernetes.io/projected/ac934633-f79c-4b67-8f34-2563d57d7ee2-kube-api-access-mg5f6\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.891152 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891156 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-serving-cert\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.891370 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891176 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-trusted-ca-bundle\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.891370 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891202 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-service-ca\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.891370 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891325 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-config\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.891484 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891371 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-oauth-config\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.891484 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.891392 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-oauth-serving-cert\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992558 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992480 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg5f6\" (UniqueName: \"kubernetes.io/projected/ac934633-f79c-4b67-8f34-2563d57d7ee2-kube-api-access-mg5f6\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992558 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-serving-cert\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992558 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992539 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-trusted-ca-bundle\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992778 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992661 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-service-ca\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992778 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992761 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-config\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992847 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992802 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-oauth-config\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.992847 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.992825 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-oauth-serving-cert\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.993451 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.993423 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-service-ca\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.993583 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.993562 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-oauth-serving-cert\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.993583 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.993526 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-trusted-ca-bundle\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.993665 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.993466 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-config\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.995510 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.995484 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-oauth-config\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:43.995598 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:43.995537 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-serving-cert\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:44.002307 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:44.002267 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg5f6\" (UniqueName: \"kubernetes.io/projected/ac934633-f79c-4b67-8f34-2563d57d7ee2-kube-api-access-mg5f6\") pod \"console-75b5597c6d-w7c9v\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:44.073465 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:44.073426 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:44.273052 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:44.272966 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b5597c6d-w7c9v"] Jul 15 13:52:44.276966 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:52:44.276938 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac934633_f79c_4b67_8f34_2563d57d7ee2.slice/crio-ffa4ba69968b130f40662982d544919a3c300908086c102046fe1545256a8ac5 WatchSource:0}: Error finding container ffa4ba69968b130f40662982d544919a3c300908086c102046fe1545256a8ac5: Status 404 returned error can't find the container with id ffa4ba69968b130f40662982d544919a3c300908086c102046fe1545256a8ac5 Jul 15 13:52:44.684613 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:44.684574 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b5597c6d-w7c9v" event={"ID":"ac934633-f79c-4b67-8f34-2563d57d7ee2","Type":"ContainerStarted","Data":"ffa4ba69968b130f40662982d544919a3c300908086c102046fe1545256a8ac5"} Jul 15 13:52:48.704795 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:48.704759 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b5597c6d-w7c9v" event={"ID":"ac934633-f79c-4b67-8f34-2563d57d7ee2","Type":"ContainerStarted","Data":"40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8"} Jul 15 13:52:48.727748 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:48.727703 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75b5597c6d-w7c9v" podStartSLOduration=2.28652196 podStartE2EDuration="5.72768559s" totalImagesPullingTime="3.44116363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:52:48.725877206 +0000 UTC m=+107.163485998" watchObservedRunningTime="2026-07-15 13:52:48.72768559 +0000 UTC m=+107.165294381" Jul 15 13:52:51.716405 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:51.716288 2584 generic.go:356] "Generic (PLEG): container finished" podID="9cd1e854-5f7e-44d5-bc9c-91fe4c24e083" containerID="635b0a378b37e639a318c6a715157c4da408b16123271214594e5741de38c032" exitCode=0 Jul 15 13:52:51.716405 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:51.716327 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" event={"ID":"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083","Type":"ContainerDied","Data":"635b0a378b37e639a318c6a715157c4da408b16123271214594e5741de38c032"} Jul 15 13:52:51.716905 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:51.716735 2584 scope.go:117] "RemoveContainer" containerID="635b0a378b37e639a318c6a715157c4da408b16123271214594e5741de38c032" Jul 15 13:52:51.718191 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:51.718169 2584 generic.go:356] "Generic (PLEG): container finished" podID="54a8e011-19a1-4f5e-a00d-e7f937aab4d2" containerID="ec448b6613c65b47c30cd842eeb6c139c165d3c131db7f7b5e772475be4a369c" exitCode=0 Jul 15 13:52:51.718277 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:51.718245 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" event={"ID":"54a8e011-19a1-4f5e-a00d-e7f937aab4d2","Type":"ContainerDied","Data":"ec448b6613c65b47c30cd842eeb6c139c165d3c131db7f7b5e772475be4a369c"} Jul 15 13:52:51.718552 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:51.718536 2584 scope.go:117] "RemoveContainer" containerID="ec448b6613c65b47c30cd842eeb6c139c165d3c131db7f7b5e772475be4a369c" Jul 15 13:52:52.724338 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:52.724272 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dtjsl" event={"ID":"9cd1e854-5f7e-44d5-bc9c-91fe4c24e083","Type":"ContainerStarted","Data":"3410b78355d1b02c5dc961150d9170648bc35d379fa63e8d15f453d16140ab75"} Jul 15 13:52:52.726193 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:52.726170 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85mvn" event={"ID":"54a8e011-19a1-4f5e-a00d-e7f937aab4d2","Type":"ContainerStarted","Data":"b3cf8c3d4810287b5ab97a410c267984defc08c5d107b7a1fd26bdccea551882"} Jul 15 13:52:54.073687 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:54.073658 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:54.074171 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:54.073733 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:54.078625 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:54.078603 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:52:54.737658 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:52:54.737630 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:53:16.811793 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:53:16.811759 2584 generic.go:356] "Generic (PLEG): container finished" podID="47accf41-e456-4b19-9cb2-edb9c0369ac2" containerID="b9eb0c9fd6fdb2dac3e7f027152465198102273bd117473ef8ad4459f499d80d" exitCode=0 Jul 15 13:53:16.812147 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:53:16.811835 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" event={"ID":"47accf41-e456-4b19-9cb2-edb9c0369ac2","Type":"ContainerDied","Data":"b9eb0c9fd6fdb2dac3e7f027152465198102273bd117473ef8ad4459f499d80d"} Jul 15 13:53:16.812393 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:53:16.812152 2584 scope.go:117] "RemoveContainer" containerID="b9eb0c9fd6fdb2dac3e7f027152465198102273bd117473ef8ad4459f499d80d" Jul 15 13:53:17.819143 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:53:17.819105 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ddh9" event={"ID":"47accf41-e456-4b19-9cb2-edb9c0369ac2","Type":"ContainerStarted","Data":"7d99d82c4f6a2d9a57ef833eecb452c0d3ff503e26804667602e505c9bd99a1b"} Jul 15 13:53:56.408345 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:53:56.408245 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75b5597c6d-w7c9v"] Jul 15 13:54:21.429220 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.429158 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75b5597c6d-w7c9v" podUID="ac934633-f79c-4b67-8f34-2563d57d7ee2" containerName="console" containerID="cri-o://40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8" gracePeriod=15 Jul 15 13:54:21.673911 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.673885 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:54:21.777025 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.776925 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-serving-cert\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777025 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.776989 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-config\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777253 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777049 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-oauth-config\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777253 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777074 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg5f6\" (UniqueName: \"kubernetes.io/projected/ac934633-f79c-4b67-8f34-2563d57d7ee2-kube-api-access-mg5f6\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777253 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777101 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-trusted-ca-bundle\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777253 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777137 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-service-ca\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777253 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777161 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-oauth-serving-cert\") pod \"ac934633-f79c-4b67-8f34-2563d57d7ee2\" (UID: \"ac934633-f79c-4b67-8f34-2563d57d7ee2\") " Jul 15 13:54:21.777521 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777498 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-config" (OuterVolumeSpecName: "console-config") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:54:21.777679 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777645 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:54:21.777777 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777651 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:54:21.777777 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.777718 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:54:21.779310 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.779275 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:54:21.779663 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.779635 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac934633-f79c-4b67-8f34-2563d57d7ee2-kube-api-access-mg5f6" (OuterVolumeSpecName: "kube-api-access-mg5f6") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "kube-api-access-mg5f6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:54:21.779721 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.779649 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac934633-f79c-4b67-8f34-2563d57d7ee2" (UID: "ac934633-f79c-4b67-8f34-2563d57d7ee2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:54:21.878700 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878669 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-config\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:21.878700 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878693 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-oauth-config\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:21.878700 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878705 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg5f6\" (UniqueName: \"kubernetes.io/projected/ac934633-f79c-4b67-8f34-2563d57d7ee2-kube-api-access-mg5f6\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:21.878923 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878713 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-trusted-ca-bundle\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:21.878923 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878722 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-service-ca\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:21.878923 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878730 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac934633-f79c-4b67-8f34-2563d57d7ee2-oauth-serving-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:21.878923 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:21.878738 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac934633-f79c-4b67-8f34-2563d57d7ee2-console-serving-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:54:22.049192 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.049117 2584 generic.go:356] "Generic (PLEG): container finished" podID="ac934633-f79c-4b67-8f34-2563d57d7ee2" containerID="40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8" exitCode=0 Jul 15 13:54:22.049192 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.049166 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b5597c6d-w7c9v" Jul 15 13:54:22.049435 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.049190 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b5597c6d-w7c9v" event={"ID":"ac934633-f79c-4b67-8f34-2563d57d7ee2","Type":"ContainerDied","Data":"40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8"} Jul 15 13:54:22.049435 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.049257 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b5597c6d-w7c9v" event={"ID":"ac934633-f79c-4b67-8f34-2563d57d7ee2","Type":"ContainerDied","Data":"ffa4ba69968b130f40662982d544919a3c300908086c102046fe1545256a8ac5"} Jul 15 13:54:22.049435 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.049279 2584 scope.go:117] "RemoveContainer" containerID="40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8" Jul 15 13:54:22.057123 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.057105 2584 scope.go:117] "RemoveContainer" containerID="40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8" Jul 15 13:54:22.057420 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:54:22.057400 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8\": container with ID starting with 40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8 not found: ID does not exist" containerID="40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8" Jul 15 13:54:22.057485 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.057427 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8"} err="failed to get container status \"40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8\": rpc error: code = NotFound desc = could not find container \"40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8\": container with ID starting with 40c2a01e440359f270d8b8c02abe74842485b85a68bebfb86245ecfae7774bf8 not found: ID does not exist" Jul 15 13:54:22.071334 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.071290 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75b5597c6d-w7c9v"] Jul 15 13:54:22.076148 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:22.076127 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75b5597c6d-w7c9v"] Jul 15 13:54:24.006739 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:54:24.006707 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac934633-f79c-4b67-8f34-2563d57d7ee2" path="/var/lib/kubelet/pods/ac934633-f79c-4b67-8f34-2563d57d7ee2/volumes" Jul 15 13:55:49.345970 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.345931 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7"] Jul 15 13:55:49.346442 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.346260 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac934633-f79c-4b67-8f34-2563d57d7ee2" containerName="console" Jul 15 13:55:49.346442 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.346272 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac934633-f79c-4b67-8f34-2563d57d7ee2" containerName="console" Jul 15 13:55:49.346442 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.346362 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac934633-f79c-4b67-8f34-2563d57d7ee2" containerName="console" Jul 15 13:55:49.349147 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.349130 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.352453 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.352434 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 13:55:49.352606 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.352590 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 13:55:49.353475 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.353461 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-w84hh\"" Jul 15 13:55:49.360499 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.360476 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7"] Jul 15 13:55:49.433930 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.433887 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4nk4\" (UniqueName: \"kubernetes.io/projected/2ae1a253-077e-4c47-8f81-aed0b24184b8-kube-api-access-j4nk4\") pod \"openshift-lws-operator-fd8ccff4c-wq8q7\" (UID: \"2ae1a253-077e-4c47-8f81-aed0b24184b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.434089 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.433946 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ae1a253-077e-4c47-8f81-aed0b24184b8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wq8q7\" (UID: \"2ae1a253-077e-4c47-8f81-aed0b24184b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.534391 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.534351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ae1a253-077e-4c47-8f81-aed0b24184b8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wq8q7\" (UID: \"2ae1a253-077e-4c47-8f81-aed0b24184b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.534580 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.534435 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4nk4\" (UniqueName: \"kubernetes.io/projected/2ae1a253-077e-4c47-8f81-aed0b24184b8-kube-api-access-j4nk4\") pod \"openshift-lws-operator-fd8ccff4c-wq8q7\" (UID: \"2ae1a253-077e-4c47-8f81-aed0b24184b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.534751 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.534729 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ae1a253-077e-4c47-8f81-aed0b24184b8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wq8q7\" (UID: \"2ae1a253-077e-4c47-8f81-aed0b24184b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.545594 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.545567 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4nk4\" (UniqueName: \"kubernetes.io/projected/2ae1a253-077e-4c47-8f81-aed0b24184b8-kube-api-access-j4nk4\") pod \"openshift-lws-operator-fd8ccff4c-wq8q7\" (UID: \"2ae1a253-077e-4c47-8f81-aed0b24184b8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.657907 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.657811 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" Jul 15 13:55:49.783383 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:49.783355 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7"] Jul 15 13:55:49.787637 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:55:49.787610 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ae1a253_077e_4c47_8f81_aed0b24184b8.slice/crio-d70fb2f65a0485cd75c94ac612d79084456617c9c746e6dc7f10eae2ad029ff2 WatchSource:0}: Error finding container d70fb2f65a0485cd75c94ac612d79084456617c9c746e6dc7f10eae2ad029ff2: Status 404 returned error can't find the container with id d70fb2f65a0485cd75c94ac612d79084456617c9c746e6dc7f10eae2ad029ff2 Jul 15 13:55:50.375811 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:50.375778 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" event={"ID":"2ae1a253-077e-4c47-8f81-aed0b24184b8","Type":"ContainerStarted","Data":"d70fb2f65a0485cd75c94ac612d79084456617c9c746e6dc7f10eae2ad029ff2"} Jul 15 13:55:53.396610 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:53.396573 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" event={"ID":"2ae1a253-077e-4c47-8f81-aed0b24184b8","Type":"ContainerStarted","Data":"e24284af63a958f54f6465ccf059d446dfa28745612eb07d9bc1083ec3063e15"} Jul 15 13:55:53.419560 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:55:53.419510 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wq8q7" podStartSLOduration=1.527881424 podStartE2EDuration="4.419495494s" totalImagesPullingTime="2.89161407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:55:53.41787729 +0000 UTC m=+291.855486086" watchObservedRunningTime="2026-07-15 13:55:53.419495494 +0000 UTC m=+291.857104284" Jul 15 13:56:01.909392 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:01.909362 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 13:56:01.917309 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:01.917268 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 13:56:01.923315 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:01.923281 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:56:01.926333 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:01.926285 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 13:56:01.927059 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:01.927043 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 13:56:13.747186 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.747152 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4"] Jul 15 13:56:13.755462 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.755446 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.766342 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.766323 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 13:56:13.766436 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.766410 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 13:56:13.766436 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.766426 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 13:56:13.782969 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.782947 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 13:56:13.793321 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.793286 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fdjm8\"" Jul 15 13:56:13.813885 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.813856 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79cc6a28-8d13-4f87-931e-e108f64edffc-tmp\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.813990 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.813890 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/79cc6a28-8d13-4f87-931e-e108f64edffc-manifests\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.813990 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.813922 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/79cc6a28-8d13-4f87-931e-e108f64edffc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.814067 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.813990 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bwmj\" (UniqueName: \"kubernetes.io/projected/79cc6a28-8d13-4f87-931e-e108f64edffc-kube-api-access-2bwmj\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.814067 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.814029 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79cc6a28-8d13-4f87-931e-e108f64edffc-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.904774 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.904739 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4"] Jul 15 13:56:13.915237 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.915210 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/79cc6a28-8d13-4f87-931e-e108f64edffc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.915352 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.915249 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bwmj\" (UniqueName: \"kubernetes.io/projected/79cc6a28-8d13-4f87-931e-e108f64edffc-kube-api-access-2bwmj\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.915352 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.915273 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79cc6a28-8d13-4f87-931e-e108f64edffc-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.915352 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.915338 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79cc6a28-8d13-4f87-931e-e108f64edffc-tmp\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.915516 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.915375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/79cc6a28-8d13-4f87-931e-e108f64edffc-manifests\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.915776 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.915752 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/79cc6a28-8d13-4f87-931e-e108f64edffc-manifests\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.917467 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.917447 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79cc6a28-8d13-4f87-931e-e108f64edffc-tmp\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.917624 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.917605 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79cc6a28-8d13-4f87-931e-e108f64edffc-webhook-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.917684 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.917639 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/79cc6a28-8d13-4f87-931e-e108f64edffc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:13.984802 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:13.984767 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bwmj\" (UniqueName: \"kubernetes.io/projected/79cc6a28-8d13-4f87-931e-e108f64edffc-kube-api-access-2bwmj\") pod \"opendatahub-operator-controller-manager-58577d4d66-5sxk4\" (UID: \"79cc6a28-8d13-4f87-931e-e108f64edffc\") " pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:14.064649 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:14.064575 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:14.216326 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:14.216282 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4"] Jul 15 13:56:14.220985 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:14.220960 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cc6a28_8d13_4f87_931e_e108f64edffc.slice/crio-755f33848998c1403f4e7443c6f22cdec038fea7a84d300e656ad7cbc1f00291 WatchSource:0}: Error finding container 755f33848998c1403f4e7443c6f22cdec038fea7a84d300e656ad7cbc1f00291: Status 404 returned error can't find the container with id 755f33848998c1403f4e7443c6f22cdec038fea7a84d300e656ad7cbc1f00291 Jul 15 13:56:14.223048 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:14.223031 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 13:56:14.476979 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:14.476945 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" event={"ID":"79cc6a28-8d13-4f87-931e-e108f64edffc","Type":"ContainerStarted","Data":"755f33848998c1403f4e7443c6f22cdec038fea7a84d300e656ad7cbc1f00291"} Jul 15 13:56:18.495818 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:18.495780 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" event={"ID":"79cc6a28-8d13-4f87-931e-e108f64edffc","Type":"ContainerStarted","Data":"21c4da04b058814cbaa7a3687e9e2d66ce4819dad2535f7991d479b7064eda12"} Jul 15 13:56:19.501529 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:19.501491 2584 generic.go:356] "Generic (PLEG): container finished" podID="79cc6a28-8d13-4f87-931e-e108f64edffc" containerID="21c4da04b058814cbaa7a3687e9e2d66ce4819dad2535f7991d479b7064eda12" exitCode=0 Jul 15 13:56:19.501921 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:19.501586 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" event={"ID":"79cc6a28-8d13-4f87-931e-e108f64edffc","Type":"ContainerDied","Data":"21c4da04b058814cbaa7a3687e9e2d66ce4819dad2535f7991d479b7064eda12"} Jul 15 13:56:20.507433 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:20.507397 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" event={"ID":"79cc6a28-8d13-4f87-931e-e108f64edffc","Type":"ContainerStarted","Data":"f7784aa4889ef64478c911058a4536b5b953a55d29f96c2e01147aa251bb6cef"} Jul 15 13:56:20.507802 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:20.507535 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:20.532903 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:20.532851 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" podStartSLOduration=2.468483809 podStartE2EDuration="7.532833112s" totalImagesPullingTime="3.978549787s" totalInitContainerRuntime="1.085799516s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:20.531709019 +0000 UTC m=+318.969317810" watchObservedRunningTime="2026-07-15 13:56:20.532833112 +0000 UTC m=+318.970441904" Jul 15 13:56:31.513626 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:31.513591 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58577d4d66-5sxk4" Jul 15 13:56:35.444372 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.444334 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z"] Jul 15 13:56:35.448278 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.448256 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.451614 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.451593 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 13:56:35.452170 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.452154 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 13:56:35.474027 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.473994 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z"] Jul 15 13:56:35.585947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.585912 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-tls-certs\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.585947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.585946 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-tmp\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.586150 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.585971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbb54\" (UniqueName: \"kubernetes.io/projected/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-kube-api-access-xbb54\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.686434 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.686400 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-tls-certs\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.686434 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.686434 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-tmp\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.686656 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.686476 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbb54\" (UniqueName: \"kubernetes.io/projected/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-kube-api-access-xbb54\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.688610 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.688588 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-tmp\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.688744 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.688723 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-tls-certs\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.726678 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.726588 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbb54\" (UniqueName: \"kubernetes.io/projected/2e2307eb-e699-49e0-bb35-821e6d2bfa9c-kube-api-access-xbb54\") pod \"kube-auth-proxy-5b5998d775-r7g6z\" (UID: \"2e2307eb-e699-49e0-bb35-821e6d2bfa9c\") " pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.757281 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.757253 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" Jul 15 13:56:35.905681 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.905642 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z"] Jul 15 13:56:35.908938 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:35.908914 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2307eb_e699_49e0_bb35_821e6d2bfa9c.slice/crio-df817fe252edb4fdc90e89bf82887c691763300406d78c619e3ce073b55ea0df WatchSource:0}: Error finding container df817fe252edb4fdc90e89bf82887c691763300406d78c619e3ce073b55ea0df: Status 404 returned error can't find the container with id df817fe252edb4fdc90e89bf82887c691763300406d78c619e3ce073b55ea0df Jul 15 13:56:35.960009 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.959973 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bpv62"] Jul 15 13:56:35.964891 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.964872 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:35.969679 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.969648 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 13:56:35.969812 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.969697 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ldqtf\"" Jul 15 13:56:35.979556 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:35.979497 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bpv62"] Jul 15 13:56:36.089891 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.089845 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a36d1b-a345-431f-a247-25f53942bf84-cert\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.089891 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.089886 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh5mx\" (UniqueName: \"kubernetes.io/projected/11a36d1b-a345-431f-a247-25f53942bf84-kube-api-access-nh5mx\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.191179 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.191145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a36d1b-a345-431f-a247-25f53942bf84-cert\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.191368 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.191194 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh5mx\" (UniqueName: \"kubernetes.io/projected/11a36d1b-a345-431f-a247-25f53942bf84-kube-api-access-nh5mx\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.191368 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:56:36.191311 2584 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 13:56:36.191442 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:56:36.191378 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11a36d1b-a345-431f-a247-25f53942bf84-cert podName:11a36d1b-a345-431f-a247-25f53942bf84 nodeName:}" failed. No retries permitted until 2026-07-15 13:56:36.691361293 +0000 UTC m=+335.128970083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/11a36d1b-a345-431f-a247-25f53942bf84-cert") pod "odh-model-controller-7998c67dc4-bpv62" (UID: "11a36d1b-a345-431f-a247-25f53942bf84") : secret "odh-model-controller-webhook-cert" not found Jul 15 13:56:36.213622 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.213595 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh5mx\" (UniqueName: \"kubernetes.io/projected/11a36d1b-a345-431f-a247-25f53942bf84-kube-api-access-nh5mx\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.572432 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.572397 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" event={"ID":"2e2307eb-e699-49e0-bb35-821e6d2bfa9c","Type":"ContainerStarted","Data":"df817fe252edb4fdc90e89bf82887c691763300406d78c619e3ce073b55ea0df"} Jul 15 13:56:36.694504 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.694469 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a36d1b-a345-431f-a247-25f53942bf84-cert\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.697652 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.697624 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a36d1b-a345-431f-a247-25f53942bf84-cert\") pod \"odh-model-controller-7998c67dc4-bpv62\" (UID: \"11a36d1b-a345-431f-a247-25f53942bf84\") " pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:36.875179 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:36.875141 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:37.075706 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.075668 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bpv62"] Jul 15 13:56:37.079872 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:37.079840 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a36d1b_a345_431f_a247_25f53942bf84.slice/crio-748dde735065340236dba3efb71c72756b0dd89ce81e0669b4cb73c53e9ef5d4 WatchSource:0}: Error finding container 748dde735065340236dba3efb71c72756b0dd89ce81e0669b4cb73c53e9ef5d4: Status 404 returned error can't find the container with id 748dde735065340236dba3efb71c72756b0dd89ce81e0669b4cb73c53e9ef5d4 Jul 15 13:56:37.579249 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.579190 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" event={"ID":"11a36d1b-a345-431f-a247-25f53942bf84","Type":"ContainerStarted","Data":"748dde735065340236dba3efb71c72756b0dd89ce81e0669b4cb73c53e9ef5d4"} Jul 15 13:56:37.630572 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.630536 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fgj4p"] Jul 15 13:56:37.635120 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.635088 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:37.638932 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.638909 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 13:56:37.646829 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.646762 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fgj4p"] Jul 15 13:56:37.699860 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.699825 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7"] Jul 15 13:56:37.705328 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.703888 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.705328 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.705253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zvt5\" (UniqueName: \"kubernetes.io/projected/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-kube-api-access-6zvt5\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:37.706165 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.705606 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:37.707282 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.707255 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 13:56:37.707534 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.707505 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 13:56:37.707729 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.707716 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 13:56:37.707967 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.707948 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 13:56:37.708116 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.708100 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bkc7d\"" Jul 15 13:56:37.728073 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.728048 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7"] Jul 15 13:56:37.806034 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806003 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806039 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-cacerts\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806103 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/975871c6-785f-4cff-a30d-a408ccb7530f-local-certs\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806137 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806206 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:37.806440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806229 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806271 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzwm8\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-kube-api-access-wzwm8\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806319 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-istio-token\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.806440 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:56:37.806335 2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 13:56:37.806440 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.806405 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zvt5\" (UniqueName: \"kubernetes.io/projected/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-kube-api-access-6zvt5\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:37.806440 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:56:37.806425 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-tls-certs podName:1f37dd4f-e424-4acd-a5b6-3eafe51a31e3 nodeName:}" failed. No retries permitted until 2026-07-15 13:56:38.306402054 +0000 UTC m=+336.744010825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-tls-certs") pod "model-serving-api-54d89bd5fc-fgj4p" (UID: "1f37dd4f-e424-4acd-a5b6-3eafe51a31e3") : secret "model-serving-api-tls" not found Jul 15 13:56:37.816768 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.816741 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zvt5\" (UniqueName: \"kubernetes.io/projected/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-kube-api-access-6zvt5\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:37.907667 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907635 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-cacerts\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.907855 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907694 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/975871c6-785f-4cff-a30d-a408ccb7530f-local-certs\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.907855 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907722 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.907855 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907797 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.907855 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907838 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzwm8\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-kube-api-access-wzwm8\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.908060 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907871 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-istio-token\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.908060 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.907926 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.908683 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.908653 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.910995 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.910966 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/975871c6-785f-4cff-a30d-a408ccb7530f-local-certs\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.911376 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.911284 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-cacerts\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.911729 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.911674 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.912462 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.912411 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.917222 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.917194 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-istio-token\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:37.917873 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:37.917813 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzwm8\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-kube-api-access-wzwm8\") pod \"istiod-openshift-gateway-7574459656-rdtz7\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:38.015259 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:38.015163 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:38.312036 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:38.311941 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:38.314951 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:38.314914 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f37dd4f-e424-4acd-a5b6-3eafe51a31e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fgj4p\" (UID: \"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:38.549085 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:38.549046 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:38.931089 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:38.930769 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fgj4p"] Jul 15 13:56:38.937527 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:38.937423 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f37dd4f_e424_4acd_a5b6_3eafe51a31e3.slice/crio-9b66554f83f0693a655681315ad233b0a81c46277eb395ac098e18902e365ef0 WatchSource:0}: Error finding container 9b66554f83f0693a655681315ad233b0a81c46277eb395ac098e18902e365ef0: Status 404 returned error can't find the container with id 9b66554f83f0693a655681315ad233b0a81c46277eb395ac098e18902e365ef0 Jul 15 13:56:38.966584 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:38.966553 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7"] Jul 15 13:56:38.969183 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:38.969138 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975871c6_785f_4cff_a30d_a408ccb7530f.slice/crio-a4a49bcda0af0fb1fbccfe992853fba0b7e672a0ff3b660e96a6ff6ce216a82b WatchSource:0}: Error finding container a4a49bcda0af0fb1fbccfe992853fba0b7e672a0ff3b660e96a6ff6ce216a82b: Status 404 returned error can't find the container with id a4a49bcda0af0fb1fbccfe992853fba0b7e672a0ff3b660e96a6ff6ce216a82b Jul 15 13:56:39.595378 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:39.595339 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" event={"ID":"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3","Type":"ContainerStarted","Data":"9b66554f83f0693a655681315ad233b0a81c46277eb395ac098e18902e365ef0"} Jul 15 13:56:39.596905 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:39.596858 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" event={"ID":"975871c6-785f-4cff-a30d-a408ccb7530f","Type":"ContainerStarted","Data":"a4a49bcda0af0fb1fbccfe992853fba0b7e672a0ff3b660e96a6ff6ce216a82b"} Jul 15 13:56:39.598562 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:39.598525 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" event={"ID":"2e2307eb-e699-49e0-bb35-821e6d2bfa9c","Type":"ContainerStarted","Data":"e82e1e011e35165fcb544bc1532233da34f3e1bc0bdc0d24dcb121a320d5c5d6"} Jul 15 13:56:39.622365 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:39.622291 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b5998d775-r7g6z" podStartSLOduration=1.725706821 podStartE2EDuration="4.622273219s" totalImagesPullingTime="2.896566398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:39.620914912 +0000 UTC m=+338.058523705" watchObservedRunningTime="2026-07-15 13:56:39.622273219 +0000 UTC m=+338.059882010" Jul 15 13:56:40.607326 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:40.607255 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" event={"ID":"11a36d1b-a345-431f-a247-25f53942bf84","Type":"ContainerStarted","Data":"0ea4795dfc744f0af0f9037de78cf1127b21519354b3b72191874fb4223b2c08"} Jul 15 13:56:40.607769 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:40.607472 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:40.625966 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:40.625912 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" podStartSLOduration=2.202723047 podStartE2EDuration="5.625893218s" totalImagesPullingTime="3.423170171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:40.625548056 +0000 UTC m=+339.063156848" watchObservedRunningTime="2026-07-15 13:56:40.625893218 +0000 UTC m=+339.063502010" Jul 15 13:56:42.291443 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.291400 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 13:56:42.291786 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.291483 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 13:56:42.622241 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.622206 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" event={"ID":"1f37dd4f-e424-4acd-a5b6-3eafe51a31e3","Type":"ContainerStarted","Data":"2363fdd764b1fcb7a805fe02209d9e02fd007ac62dc96b8935a26605e9e6d737"} Jul 15 13:56:42.622456 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.622325 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:42.624073 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.624042 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" event={"ID":"975871c6-785f-4cff-a30d-a408ccb7530f","Type":"ContainerStarted","Data":"d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b"} Jul 15 13:56:42.624185 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.624170 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:42.642044 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.641995 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" podStartSLOduration=2.290677767 podStartE2EDuration="5.641976168s" totalImagesPullingTime="3.351298401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:42.640275937 +0000 UTC m=+341.077884731" watchObservedRunningTime="2026-07-15 13:56:42.641976168 +0000 UTC m=+341.079584959" Jul 15 13:56:42.664612 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:42.664566 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" podStartSLOduration=2.346956842 podStartE2EDuration="5.664550726s" totalImagesPullingTime="3.317593884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:42.663619319 +0000 UTC m=+341.101228112" watchObservedRunningTime="2026-07-15 13:56:42.664550726 +0000 UTC m=+341.102159516" Jul 15 13:56:43.631051 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:43.631014 2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-rdtz7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 13:56:43.631403 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:43.631074 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" podUID="975871c6-785f-4cff-a30d-a408ccb7530f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 13:56:46.630755 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:46.630723 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:49.169158 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.169111 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m"] Jul 15 13:56:49.174132 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.174111 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-78pn9"] Jul 15 13:56:49.174248 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.174235 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.177266 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.177249 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.190688 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.190668 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nf2hr\"" Jul 15 13:56:49.194940 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.194922 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 13:56:49.195841 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.195823 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 13:56:49.196087 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.196073 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cvgds\"" Jul 15 13:56:49.209238 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.209218 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-78pn9"] Jul 15 13:56:49.224922 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.224901 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m"] Jul 15 13:56:49.312209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.312185 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64f00680-3358-41e7-80f2-efcd392d726e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rw87m\" (UID: \"64f00680-3358-41e7-80f2-efcd392d726e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.312377 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.312219 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa4ca8eb-ad72-497a-9d07-1570602a84ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-78pn9\" (UID: \"aa4ca8eb-ad72-497a-9d07-1570602a84ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.312377 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.312257 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bclq\" (UniqueName: \"kubernetes.io/projected/64f00680-3358-41e7-80f2-efcd392d726e-kube-api-access-4bclq\") pod \"llmisvc-controller-manager-777c6bd7d5-rw87m\" (UID: \"64f00680-3358-41e7-80f2-efcd392d726e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.312377 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.312288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbwt5\" (UniqueName: \"kubernetes.io/projected/aa4ca8eb-ad72-497a-9d07-1570602a84ab-kube-api-access-nbwt5\") pod \"kserve-controller-manager-d6f4b65b8-78pn9\" (UID: \"aa4ca8eb-ad72-497a-9d07-1570602a84ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.413380 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.413354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bclq\" (UniqueName: \"kubernetes.io/projected/64f00680-3358-41e7-80f2-efcd392d726e-kube-api-access-4bclq\") pod \"llmisvc-controller-manager-777c6bd7d5-rw87m\" (UID: \"64f00680-3358-41e7-80f2-efcd392d726e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.413519 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.413383 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbwt5\" (UniqueName: \"kubernetes.io/projected/aa4ca8eb-ad72-497a-9d07-1570602a84ab-kube-api-access-nbwt5\") pod \"kserve-controller-manager-d6f4b65b8-78pn9\" (UID: \"aa4ca8eb-ad72-497a-9d07-1570602a84ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.413519 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.413423 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64f00680-3358-41e7-80f2-efcd392d726e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rw87m\" (UID: \"64f00680-3358-41e7-80f2-efcd392d726e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.413519 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.413459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa4ca8eb-ad72-497a-9d07-1570602a84ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-78pn9\" (UID: \"aa4ca8eb-ad72-497a-9d07-1570602a84ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.415899 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.415875 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64f00680-3358-41e7-80f2-efcd392d726e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rw87m\" (UID: \"64f00680-3358-41e7-80f2-efcd392d726e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.415995 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.415882 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa4ca8eb-ad72-497a-9d07-1570602a84ab-cert\") pod \"kserve-controller-manager-d6f4b65b8-78pn9\" (UID: \"aa4ca8eb-ad72-497a-9d07-1570602a84ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.459730 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.459681 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbwt5\" (UniqueName: \"kubernetes.io/projected/aa4ca8eb-ad72-497a-9d07-1570602a84ab-kube-api-access-nbwt5\") pod \"kserve-controller-manager-d6f4b65b8-78pn9\" (UID: \"aa4ca8eb-ad72-497a-9d07-1570602a84ab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.459730 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.459685 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bclq\" (UniqueName: \"kubernetes.io/projected/64f00680-3358-41e7-80f2-efcd392d726e-kube-api-access-4bclq\") pod \"llmisvc-controller-manager-777c6bd7d5-rw87m\" (UID: \"64f00680-3358-41e7-80f2-efcd392d726e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.484471 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.484451 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:49.490483 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.490468 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:49.642477 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.642446 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m"] Jul 15 13:56:49.645204 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:49.645177 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod64f00680_3358_41e7_80f2_efcd392d726e.slice/crio-22b310c48e8c95a7040d3b41f8d233659d19cb9f5b90e9bff8795214efe3d8ff WatchSource:0}: Error finding container 22b310c48e8c95a7040d3b41f8d233659d19cb9f5b90e9bff8795214efe3d8ff: Status 404 returned error can't find the container with id 22b310c48e8c95a7040d3b41f8d233659d19cb9f5b90e9bff8795214efe3d8ff Jul 15 13:56:49.649935 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.649910 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-78pn9"] Jul 15 13:56:49.651501 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:56:49.651480 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4ca8eb_ad72_497a_9d07_1570602a84ab.slice/crio-333d85a6607b7cc22d01fbbdf77feaad274ece15b02ad00233d1c5f4d6608218 WatchSource:0}: Error finding container 333d85a6607b7cc22d01fbbdf77feaad274ece15b02ad00233d1c5f4d6608218: Status 404 returned error can't find the container with id 333d85a6607b7cc22d01fbbdf77feaad274ece15b02ad00233d1c5f4d6608218 Jul 15 13:56:49.655477 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:49.655452 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" event={"ID":"64f00680-3358-41e7-80f2-efcd392d726e","Type":"ContainerStarted","Data":"22b310c48e8c95a7040d3b41f8d233659d19cb9f5b90e9bff8795214efe3d8ff"} Jul 15 13:56:50.190763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.190720 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7"] Jul 15 13:56:50.191226 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.190891 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" podUID="975871c6-785f-4cff-a30d-a408ccb7530f" containerName="discovery" containerID="cri-o://d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b" gracePeriod=30 Jul 15 13:56:50.446212 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.446153 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:50.524702 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.524675 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-dns-cert\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.524857 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.524737 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzwm8\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-kube-api-access-wzwm8\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.524857 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.524767 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-cacerts\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.524992 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.524863 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-istio-token\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.524992 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.524912 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-ca-configmap\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.524992 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.524967 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/975871c6-785f-4cff-a30d-a408ccb7530f-local-certs\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.525119 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.525006 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-kubeconfig\") pod \"975871c6-785f-4cff-a30d-a408ccb7530f\" (UID: \"975871c6-785f-4cff-a30d-a408ccb7530f\") " Jul 15 13:56:50.525328 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.525277 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:56:50.527197 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.527159 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:56:50.527197 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.527172 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/975871c6-785f-4cff-a30d-a408ccb7530f-local-certs" (OuterVolumeSpecName: "local-certs") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:56:50.527401 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.527376 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:56:50.527497 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.527398 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-cacerts" (OuterVolumeSpecName: "cacerts") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 13:56:50.527497 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.527411 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-istio-token" (OuterVolumeSpecName: "istio-token") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:56:50.527568 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.527536 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-kube-api-access-wzwm8" (OuterVolumeSpecName: "kube-api-access-wzwm8") pod "975871c6-785f-4cff-a30d-a408ccb7530f" (UID: "975871c6-785f-4cff-a30d-a408ccb7530f"). InnerVolumeSpecName "kube-api-access-wzwm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:56:50.626195 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626159 2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-dns-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.626195 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626197 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzwm8\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-kube-api-access-wzwm8\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.626445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626211 2584 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-cacerts\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.626445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626219 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975871c6-785f-4cff-a30d-a408ccb7530f-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.626445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626229 2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/975871c6-785f-4cff-a30d-a408ccb7530f-istio-csr-ca-configmap\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.626445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626238 2584 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/975871c6-785f-4cff-a30d-a408ccb7530f-local-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.626445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.626246 2584 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/975871c6-785f-4cff-a30d-a408ccb7530f-istio-kubeconfig\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:56:50.661374 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.661339 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" event={"ID":"aa4ca8eb-ad72-497a-9d07-1570602a84ab","Type":"ContainerStarted","Data":"333d85a6607b7cc22d01fbbdf77feaad274ece15b02ad00233d1c5f4d6608218"} Jul 15 13:56:50.662803 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.662763 2584 generic.go:356] "Generic (PLEG): container finished" podID="975871c6-785f-4cff-a30d-a408ccb7530f" containerID="d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b" exitCode=0 Jul 15 13:56:50.662944 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.662806 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" Jul 15 13:56:50.662944 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.662836 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" event={"ID":"975871c6-785f-4cff-a30d-a408ccb7530f","Type":"ContainerDied","Data":"d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b"} Jul 15 13:56:50.662944 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.662874 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7" event={"ID":"975871c6-785f-4cff-a30d-a408ccb7530f","Type":"ContainerDied","Data":"a4a49bcda0af0fb1fbccfe992853fba0b7e672a0ff3b660e96a6ff6ce216a82b"} Jul 15 13:56:50.662944 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.662901 2584 scope.go:117] "RemoveContainer" containerID="d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b" Jul 15 13:56:50.673383 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.673363 2584 scope.go:117] "RemoveContainer" containerID="d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b" Jul 15 13:56:50.673662 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:56:50.673638 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b\": container with ID starting with d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b not found: ID does not exist" containerID="d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b" Jul 15 13:56:50.673800 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.673669 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b"} err="failed to get container status \"d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b\": rpc error: code = NotFound desc = could not find container \"d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b\": container with ID starting with d489ca75c9b81c17b264a175ec2463c364bc2ce015c015ca3a18d16ee1a5dd4b not found: ID does not exist" Jul 15 13:56:50.692431 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.692406 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7"] Jul 15 13:56:50.693983 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:50.693956 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rdtz7"] Jul 15 13:56:51.618703 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:51.618670 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bpv62" Jul 15 13:56:52.008687 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:52.008602 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="975871c6-785f-4cff-a30d-a408ccb7530f" path="/var/lib/kubelet/pods/975871c6-785f-4cff-a30d-a408ccb7530f/volumes" Jul 15 13:56:52.676073 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:52.676035 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" event={"ID":"aa4ca8eb-ad72-497a-9d07-1570602a84ab","Type":"ContainerStarted","Data":"cbb34f791c77a7b7b56dd0ee3a2f71df6c68d96e7b386235559a69ad0533758e"} Jul 15 13:56:52.676579 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:52.676187 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:56:52.723443 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:52.723396 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" podStartSLOduration=1.098480165 podStartE2EDuration="3.723377293s" totalImagesPullingTime="2.624897128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:52.721384679 +0000 UTC m=+351.158993470" watchObservedRunningTime="2026-07-15 13:56:52.723377293 +0000 UTC m=+351.160986083" Jul 15 13:56:53.633584 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:53.633552 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fgj4p" Jul 15 13:56:54.687378 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:54.687342 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" event={"ID":"64f00680-3358-41e7-80f2-efcd392d726e","Type":"ContainerStarted","Data":"88c95d7a318d3e8f9a8f0cde89b46cd33b651f2eab731cb7bf67244ba8738144"} Jul 15 13:56:54.687752 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:54.687461 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:56:54.706959 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:56:54.706911 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" podStartSLOduration=1.147866164 podStartE2EDuration="5.70689519s" totalImagesPullingTime="4.559029026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:56:54.706313836 +0000 UTC m=+353.143922627" watchObservedRunningTime="2026-07-15 13:56:54.70689519 +0000 UTC m=+353.144503980" Jul 15 13:57:23.686812 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:23.686785 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-78pn9" Jul 15 13:57:25.694103 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:25.694066 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rw87m" Jul 15 13:57:36.888974 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.888941 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmd7g"] Jul 15 13:57:36.889343 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.889307 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975871c6-785f-4cff-a30d-a408ccb7530f" containerName="discovery" Jul 15 13:57:36.889343 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.889321 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="975871c6-785f-4cff-a30d-a408ccb7530f" containerName="discovery" Jul 15 13:57:36.889418 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.889378 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="975871c6-785f-4cff-a30d-a408ccb7530f" containerName="discovery" Jul 15 13:57:36.895129 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.895111 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:36.902323 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.902290 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 13:57:36.903333 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.903290 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 13:57:36.903436 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.903290 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-t4qss\"" Jul 15 13:57:36.916191 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:36.916165 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmd7g"] Jul 15 13:57:37.027975 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.027942 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlwzj\" (UniqueName: \"kubernetes.io/projected/64460535-cbd0-485c-ae50-927fd69604c5-kube-api-access-rlwzj\") pod \"kuadrant-operator-catalog-mmd7g\" (UID: \"64460535-cbd0-485c-ae50-927fd69604c5\") " pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:37.112613 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.112577 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmd7g"] Jul 15 13:57:37.112857 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:57:37.112837 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rlwzj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" podUID="64460535-cbd0-485c-ae50-927fd69604c5" Jul 15 13:57:37.128673 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.128642 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlwzj\" (UniqueName: \"kubernetes.io/projected/64460535-cbd0-485c-ae50-927fd69604c5-kube-api-access-rlwzj\") pod \"kuadrant-operator-catalog-mmd7g\" (UID: \"64460535-cbd0-485c-ae50-927fd69604c5\") " pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:37.137814 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.137788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlwzj\" (UniqueName: \"kubernetes.io/projected/64460535-cbd0-485c-ae50-927fd69604c5-kube-api-access-rlwzj\") pod \"kuadrant-operator-catalog-mmd7g\" (UID: \"64460535-cbd0-485c-ae50-927fd69604c5\") " pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:37.320548 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.320462 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pl69n"] Jul 15 13:57:37.323727 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.323705 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:37.333944 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.333919 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pl69n"] Jul 15 13:57:37.431083 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.431046 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlvjt\" (UniqueName: \"kubernetes.io/projected/03642870-f19d-4884-9373-d751cd253ea5-kube-api-access-vlvjt\") pod \"kuadrant-operator-catalog-pl69n\" (UID: \"03642870-f19d-4884-9373-d751cd253ea5\") " pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:37.535492 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.532823 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlvjt\" (UniqueName: \"kubernetes.io/projected/03642870-f19d-4884-9373-d751cd253ea5-kube-api-access-vlvjt\") pod \"kuadrant-operator-catalog-pl69n\" (UID: \"03642870-f19d-4884-9373-d751cd253ea5\") " pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:37.542446 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.542423 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlvjt\" (UniqueName: \"kubernetes.io/projected/03642870-f19d-4884-9373-d751cd253ea5-kube-api-access-vlvjt\") pod \"kuadrant-operator-catalog-pl69n\" (UID: \"03642870-f19d-4884-9373-d751cd253ea5\") " pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:37.633580 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.633551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:37.765049 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.764994 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pl69n"] Jul 15 13:57:37.767630 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:57:37.767594 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03642870_f19d_4884_9373_d751cd253ea5.slice/crio-38f21477693516ff36d6422278254e7eba417812954c90757719a203054158bb WatchSource:0}: Error finding container 38f21477693516ff36d6422278254e7eba417812954c90757719a203054158bb: Status 404 returned error can't find the container with id 38f21477693516ff36d6422278254e7eba417812954c90757719a203054158bb Jul 15 13:57:37.877213 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.877188 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:37.877371 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.877173 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" event={"ID":"03642870-f19d-4884-9373-d751cd253ea5","Type":"ContainerStarted","Data":"38f21477693516ff36d6422278254e7eba417812954c90757719a203054158bb"} Jul 15 13:57:37.884996 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:37.884945 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:38.037243 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:38.037214 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlwzj\" (UniqueName: \"kubernetes.io/projected/64460535-cbd0-485c-ae50-927fd69604c5-kube-api-access-rlwzj\") pod \"64460535-cbd0-485c-ae50-927fd69604c5\" (UID: \"64460535-cbd0-485c-ae50-927fd69604c5\") " Jul 15 13:57:38.039245 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:38.039224 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64460535-cbd0-485c-ae50-927fd69604c5-kube-api-access-rlwzj" (OuterVolumeSpecName: "kube-api-access-rlwzj") pod "64460535-cbd0-485c-ae50-927fd69604c5" (UID: "64460535-cbd0-485c-ae50-927fd69604c5"). InnerVolumeSpecName "kube-api-access-rlwzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:57:38.138870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:38.138797 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlwzj\" (UniqueName: \"kubernetes.io/projected/64460535-cbd0-485c-ae50-927fd69604c5-kube-api-access-rlwzj\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:57:38.882598 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:38.882563 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmd7g" Jul 15 13:57:38.918229 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:38.918193 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmd7g"] Jul 15 13:57:38.921816 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:38.921778 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmd7g"] Jul 15 13:57:39.887963 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:39.887921 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" event={"ID":"03642870-f19d-4884-9373-d751cd253ea5","Type":"ContainerStarted","Data":"3b3d595644c461428b6cf1371fc9d188caef1f84cc76f7483b1c05b770b511df"} Jul 15 13:57:39.923134 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:39.923075 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" podStartSLOduration=1.009308238 podStartE2EDuration="2.923056214s" totalImagesPullingTime="1.913747976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:57:39.921775236 +0000 UTC m=+398.359384027" watchObservedRunningTime="2026-07-15 13:57:39.923056214 +0000 UTC m=+398.360665004" Jul 15 13:57:40.007273 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:40.007181 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64460535-cbd0-485c-ae50-927fd69604c5" path="/var/lib/kubelet/pods/64460535-cbd0-485c-ae50-927fd69604c5/volumes" Jul 15 13:57:47.634161 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:47.634115 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:47.634584 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:47.634191 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:47.657276 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:47.657249 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:57:47.943229 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:57:47.943160 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pl69n" Jul 15 13:58:18.793725 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.793691 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk"] Jul 15 13:58:18.802029 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.802008 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:18.804831 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.804809 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-q2fmf\"" Jul 15 13:58:18.809656 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.809634 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk"] Jul 15 13:58:18.867763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.867710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l7td\" (UniqueName: \"kubernetes.io/projected/5da529bc-a05d-4c68-81f7-37955bd42359-kube-api-access-4l7td\") pod \"authorino-operator-75cc8d5cb7-hq6vk\" (UID: \"5da529bc-a05d-4c68-81f7-37955bd42359\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:18.968692 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.968641 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l7td\" (UniqueName: \"kubernetes.io/projected/5da529bc-a05d-4c68-81f7-37955bd42359-kube-api-access-4l7td\") pod \"authorino-operator-75cc8d5cb7-hq6vk\" (UID: \"5da529bc-a05d-4c68-81f7-37955bd42359\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:18.978635 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:18.978603 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l7td\" (UniqueName: \"kubernetes.io/projected/5da529bc-a05d-4c68-81f7-37955bd42359-kube-api-access-4l7td\") pod \"authorino-operator-75cc8d5cb7-hq6vk\" (UID: \"5da529bc-a05d-4c68-81f7-37955bd42359\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:19.112108 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:19.112081 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:19.246121 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:19.246092 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk"] Jul 15 13:58:19.250045 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:58:19.250020 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da529bc_a05d_4c68_81f7_37955bd42359.slice/crio-4b33d9ae902664a6328f2d1b106730f7e2ac4c2cf546057080affff527a43d18 WatchSource:0}: Error finding container 4b33d9ae902664a6328f2d1b106730f7e2ac4c2cf546057080affff527a43d18: Status 404 returned error can't find the container with id 4b33d9ae902664a6328f2d1b106730f7e2ac4c2cf546057080affff527a43d18 Jul 15 13:58:20.059279 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:20.059242 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" event={"ID":"5da529bc-a05d-4c68-81f7-37955bd42359","Type":"ContainerStarted","Data":"4b33d9ae902664a6328f2d1b106730f7e2ac4c2cf546057080affff527a43d18"} Jul 15 13:58:21.777662 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.777624 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2"] Jul 15 13:58:21.781014 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.780991 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:21.783696 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.783678 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 13:58:21.783788 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.783735 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xt6t4\"" Jul 15 13:58:21.793565 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.793543 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2"] Jul 15 13:58:21.895754 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.895661 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jng9q\" (UniqueName: \"kubernetes.io/projected/c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7-kube-api-access-jng9q\") pod \"dns-operator-controller-manager-864dd78c4c-fzdk2\" (UID: \"c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:21.996970 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:21.996936 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jng9q\" (UniqueName: \"kubernetes.io/projected/c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7-kube-api-access-jng9q\") pod \"dns-operator-controller-manager-864dd78c4c-fzdk2\" (UID: \"c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:22.008209 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:22.008183 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jng9q\" (UniqueName: \"kubernetes.io/projected/c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7-kube-api-access-jng9q\") pod \"dns-operator-controller-manager-864dd78c4c-fzdk2\" (UID: \"c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:22.070192 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:22.070156 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" event={"ID":"5da529bc-a05d-4c68-81f7-37955bd42359","Type":"ContainerStarted","Data":"3d03c37f229620033cbb0d0d5e62087a33ed809d39034dec925428c0ea2c6fd6"} Jul 15 13:58:22.070399 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:22.070285 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:22.088992 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:22.088948 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" podStartSLOduration=1.71084931 podStartE2EDuration="4.088928483s" totalImagesPullingTime="2.378079173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:22.087275315 +0000 UTC m=+440.524884107" watchObservedRunningTime="2026-07-15 13:58:22.088928483 +0000 UTC m=+440.526537274" Jul 15 13:58:22.090416 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:22.090394 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:22.217712 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:22.217599 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2"] Jul 15 13:58:22.219849 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:58:22.219814 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0cc3ef1_49e1_45b3_8194_4f6a1d0655f7.slice/crio-2e1472e88b2f21977d314cc7009809de861cb7de205cbb86d7654e75ec54a9ff WatchSource:0}: Error finding container 2e1472e88b2f21977d314cc7009809de861cb7de205cbb86d7654e75ec54a9ff: Status 404 returned error can't find the container with id 2e1472e88b2f21977d314cc7009809de861cb7de205cbb86d7654e75ec54a9ff Jul 15 13:58:23.079774 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:23.079731 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" event={"ID":"c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7","Type":"ContainerStarted","Data":"2e1472e88b2f21977d314cc7009809de861cb7de205cbb86d7654e75ec54a9ff"} Jul 15 13:58:25.091028 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.090995 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" event={"ID":"c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7","Type":"ContainerStarted","Data":"2c60071f6185061bce13cfd60b0555babeec76fa8510f0228b7d9961e2e4b946"} Jul 15 13:58:25.091430 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.091172 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:25.115033 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.114990 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" podStartSLOduration=2.312336909 podStartE2EDuration="4.114976538s" totalImagesPullingTime="1.802639629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:25.113249732 +0000 UTC m=+443.550858522" watchObservedRunningTime="2026-07-15 13:58:25.114976538 +0000 UTC m=+443.552585330" Jul 15 13:58:25.267917 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.267891 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2"] Jul 15 13:58:25.271053 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.271034 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:25.274901 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.274878 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6zwft\"" Jul 15 13:58:25.287238 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.287216 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2"] Jul 15 13:58:25.326237 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.326214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcf6l\" (UniqueName: \"kubernetes.io/projected/182bca5b-f6b3-42c7-b37a-8b58fc795207-kube-api-access-mcf6l\") pod \"limitador-operator-controller-manager-859d7b78bd-9khv2\" (UID: \"182bca5b-f6b3-42c7-b37a-8b58fc795207\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:25.427026 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.426973 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcf6l\" (UniqueName: \"kubernetes.io/projected/182bca5b-f6b3-42c7-b37a-8b58fc795207-kube-api-access-mcf6l\") pod \"limitador-operator-controller-manager-859d7b78bd-9khv2\" (UID: \"182bca5b-f6b3-42c7-b37a-8b58fc795207\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:25.456729 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.456704 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcf6l\" (UniqueName: \"kubernetes.io/projected/182bca5b-f6b3-42c7-b37a-8b58fc795207-kube-api-access-mcf6l\") pod \"limitador-operator-controller-manager-859d7b78bd-9khv2\" (UID: \"182bca5b-f6b3-42c7-b37a-8b58fc795207\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:25.581273 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.581236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:25.728134 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:25.728113 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2"] Jul 15 13:58:25.729944 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:58:25.729916 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182bca5b_f6b3_42c7_b37a_8b58fc795207.slice/crio-ee1f676a01691d8929657242d268ea918278cd4eae54b4044903669b9052f138 WatchSource:0}: Error finding container ee1f676a01691d8929657242d268ea918278cd4eae54b4044903669b9052f138: Status 404 returned error can't find the container with id ee1f676a01691d8929657242d268ea918278cd4eae54b4044903669b9052f138 Jul 15 13:58:26.099347 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:26.099312 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" event={"ID":"182bca5b-f6b3-42c7-b37a-8b58fc795207","Type":"ContainerStarted","Data":"ee1f676a01691d8929657242d268ea918278cd4eae54b4044903669b9052f138"} Jul 15 13:58:28.110455 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:28.110423 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" event={"ID":"182bca5b-f6b3-42c7-b37a-8b58fc795207","Type":"ContainerStarted","Data":"2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7"} Jul 15 13:58:28.110834 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:28.110552 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:28.135061 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:28.134898 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" podStartSLOduration=1.3873918029999999 podStartE2EDuration="3.134880297s" totalImagesPullingTime="1.747488494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:28.134683527 +0000 UTC m=+446.572292321" watchObservedRunningTime="2026-07-15 13:58:28.134880297 +0000 UTC m=+446.572489088" Jul 15 13:58:30.385810 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.385777 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8"] Jul 15 13:58:30.389123 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.389107 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.395138 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.395111 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nwrzz\"" Jul 15 13:58:30.472573 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.472549 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljcwm\" (UniqueName: \"kubernetes.io/projected/806dd485-5977-47d8-b875-fe224b5cbbaa-kube-api-access-ljcwm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.472695 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.472603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/806dd485-5977-47d8-b875-fe224b5cbbaa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.521534 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.521501 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8"] Jul 15 13:58:30.573144 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.573114 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/806dd485-5977-47d8-b875-fe224b5cbbaa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.573281 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.573171 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljcwm\" (UniqueName: \"kubernetes.io/projected/806dd485-5977-47d8-b875-fe224b5cbbaa-kube-api-access-ljcwm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.573504 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.573485 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/806dd485-5977-47d8-b875-fe224b5cbbaa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.584079 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.584052 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljcwm\" (UniqueName: \"kubernetes.io/projected/806dd485-5977-47d8-b875-fe224b5cbbaa-kube-api-access-ljcwm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.699517 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.699461 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:30.836557 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:30.836498 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8"] Jul 15 13:58:30.838963 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:58:30.838939 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806dd485_5977_47d8_b875_fe224b5cbbaa.slice/crio-b82a806a73956f295fbc238fc90b6269f33465ef76931c7abea42efdad5f3073 WatchSource:0}: Error finding container b82a806a73956f295fbc238fc90b6269f33465ef76931c7abea42efdad5f3073: Status 404 returned error can't find the container with id b82a806a73956f295fbc238fc90b6269f33465ef76931c7abea42efdad5f3073 Jul 15 13:58:31.125258 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:31.125220 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" event={"ID":"806dd485-5977-47d8-b875-fe224b5cbbaa","Type":"ContainerStarted","Data":"b82a806a73956f295fbc238fc90b6269f33465ef76931c7abea42efdad5f3073"} Jul 15 13:58:33.081936 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:33.081907 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hq6vk" Jul 15 13:58:36.102059 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:36.102029 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fzdk2" Jul 15 13:58:37.156981 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:37.156944 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" event={"ID":"806dd485-5977-47d8-b875-fe224b5cbbaa","Type":"ContainerStarted","Data":"5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56"} Jul 15 13:58:37.157404 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:37.157064 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:37.187761 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:37.187714 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" podStartSLOduration=1.7630464209999999 podStartE2EDuration="7.18769932s" totalImagesPullingTime="5.424652899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:37.185977786 +0000 UTC m=+455.623586577" watchObservedRunningTime="2026-07-15 13:58:37.18769932 +0000 UTC m=+455.625308110" Jul 15 13:58:39.118057 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:39.118033 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:48.164413 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:48.164383 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:49.870711 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:49.870681 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58"] Jul 15 13:58:49.873809 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:49.873793 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:49.891691 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:49.891665 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58"] Jul 15 13:58:49.989746 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:49.989717 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58"] Jul 15 13:58:49.989925 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:49.989908 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-ls672], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" Jul 15 13:58:50.030178 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.030150 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce35658a-c467-41bc-b228-5963fdd9c484-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" (UID: \"ce35658a-c467-41bc-b228-5963fdd9c484\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.030323 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.030239 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls672\" (UniqueName: \"kubernetes.io/projected/ce35658a-c467-41bc-b228-5963fdd9c484-kube-api-access-ls672\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" (UID: \"ce35658a-c467-41bc-b228-5963fdd9c484\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.097594 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.097568 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58"] Jul 15 13:58:50.113659 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.113635 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58"] Jul 15 13:58:50.126652 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.126596 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8"] Jul 15 13:58:50.126792 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.126769 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" containerName="manager" containerID="cri-o://5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56" gracePeriod=2 Jul 15 13:58:50.131374 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.131348 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls672\" (UniqueName: \"kubernetes.io/projected/ce35658a-c467-41bc-b228-5963fdd9c484-kube-api-access-ls672\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" (UID: \"ce35658a-c467-41bc-b228-5963fdd9c484\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.132113 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.132091 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce35658a-c467-41bc-b228-5963fdd9c484-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" (UID: \"ce35658a-c467-41bc-b228-5963fdd9c484\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.132903 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.132877 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce35658a-c467-41bc-b228-5963fdd9c484-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" (UID: \"ce35658a-c467-41bc-b228-5963fdd9c484\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.133522 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.133491 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8"] Jul 15 13:58:50.134222 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.134202 2584 projected.go:196] Error preparing data for projected volume kube-api-access-ls672 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-133-21.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object Jul 15 13:58:50.134410 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.134396 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce35658a-c467-41bc-b228-5963fdd9c484-kube-api-access-ls672 podName:ce35658a-c467-41bc-b228-5963fdd9c484 nodeName:}" failed. No retries permitted until 2026-07-15 13:58:50.634369591 +0000 UTC m=+469.071978373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ls672" (UniqueName: "kubernetes.io/projected/ce35658a-c467-41bc-b228-5963fdd9c484-kube-api-access-ls672") pod "kuadrant-operator-controller-manager-55467d9f7-b8t58" (UID: "ce35658a-c467-41bc-b228-5963fdd9c484") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-133-21.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object Jul 15 13:58:50.170346 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.170322 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc"] Jul 15 13:58:50.170732 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.170715 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" containerName="manager" Jul 15 13:58:50.170793 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.170736 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" containerName="manager" Jul 15 13:58:50.170864 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.170852 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" containerName="manager" Jul 15 13:58:50.173757 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.173739 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.184444 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.184424 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb"] Jul 15 13:58:50.187455 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.187440 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.191431 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.191412 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2"] Jul 15 13:58:50.191574 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.191556 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" containerName="manager" containerID="cri-o://2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7" gracePeriod=2 Jul 15 13:58:50.199220 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.199200 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc"] Jul 15 13:58:50.202842 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.202818 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2"] Jul 15 13:58:50.213078 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.213050 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.220846 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.220822 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb"] Jul 15 13:58:50.245680 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.245649 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:50.268941 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.268920 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.285321 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.285271 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.287872 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.287840 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:50.290117 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.290092 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:50.292366 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.292347 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.294410 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.294385 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:50.301534 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.301499 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:50.334886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.334549 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ebfa81-2287-4a58-bf52-efe8660aff90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-snpmb\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.334886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.334601 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwplh\" (UniqueName: \"kubernetes.io/projected/e5ebfa81-2287-4a58-bf52-efe8660aff90-kube-api-access-lwplh\") pod \"kuadrant-operator-controller-manager-86878c769f-snpmb\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.334886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.334685 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kbchc\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.334886 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.334793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtnxr\" (UniqueName: \"kubernetes.io/projected/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-kube-api-access-gtnxr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kbchc\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.435948 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.435928 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce35658a-c467-41bc-b228-5963fdd9c484-extensions-socket-volume\") pod \"ce35658a-c467-41bc-b228-5963fdd9c484\" (UID: \"ce35658a-c467-41bc-b228-5963fdd9c484\") " Jul 15 13:58:50.436061 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436047 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtnxr\" (UniqueName: \"kubernetes.io/projected/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-kube-api-access-gtnxr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kbchc\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.436111 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436086 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ebfa81-2287-4a58-bf52-efe8660aff90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-snpmb\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.436168 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436107 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwplh\" (UniqueName: \"kubernetes.io/projected/e5ebfa81-2287-4a58-bf52-efe8660aff90-kube-api-access-lwplh\") pod \"kuadrant-operator-controller-manager-86878c769f-snpmb\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.436218 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436160 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce35658a-c467-41bc-b228-5963fdd9c484-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ce35658a-c467-41bc-b228-5963fdd9c484" (UID: "ce35658a-c467-41bc-b228-5963fdd9c484"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:58:50.436218 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436171 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kbchc\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.436324 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436264 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce35658a-c467-41bc-b228-5963fdd9c484-extensions-socket-volume\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:58:50.436324 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436281 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls672\" (UniqueName: \"kubernetes.io/projected/ce35658a-c467-41bc-b228-5963fdd9c484-kube-api-access-ls672\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:58:50.436505 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436483 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ebfa81-2287-4a58-bf52-efe8660aff90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-snpmb\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.436629 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436613 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kbchc\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.436723 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.436703 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:50.439870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.439850 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:50.440847 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.440822 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:50.443033 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.443011 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.446240 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.446210 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:50.448484 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.448456 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:50.450173 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.450154 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwplh\" (UniqueName: \"kubernetes.io/projected/e5ebfa81-2287-4a58-bf52-efe8660aff90-kube-api-access-lwplh\") pod \"kuadrant-operator-controller-manager-86878c769f-snpmb\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.450553 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.450532 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtnxr\" (UniqueName: \"kubernetes.io/projected/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-kube-api-access-gtnxr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kbchc\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.451744 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.451720 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:50.454095 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:50.454075 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:50.536676 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.536653 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljcwm\" (UniqueName: \"kubernetes.io/projected/806dd485-5977-47d8-b875-fe224b5cbbaa-kube-api-access-ljcwm\") pod \"806dd485-5977-47d8-b875-fe224b5cbbaa\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " Jul 15 13:58:50.536827 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.536687 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcf6l\" (UniqueName: \"kubernetes.io/projected/182bca5b-f6b3-42c7-b37a-8b58fc795207-kube-api-access-mcf6l\") pod \"182bca5b-f6b3-42c7-b37a-8b58fc795207\" (UID: \"182bca5b-f6b3-42c7-b37a-8b58fc795207\") " Jul 15 13:58:50.536827 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.536751 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/806dd485-5977-47d8-b875-fe224b5cbbaa-extensions-socket-volume\") pod \"806dd485-5977-47d8-b875-fe224b5cbbaa\" (UID: \"806dd485-5977-47d8-b875-fe224b5cbbaa\") " Jul 15 13:58:50.537252 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.537227 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/806dd485-5977-47d8-b875-fe224b5cbbaa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "806dd485-5977-47d8-b875-fe224b5cbbaa" (UID: "806dd485-5977-47d8-b875-fe224b5cbbaa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:58:50.538685 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.538663 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/806dd485-5977-47d8-b875-fe224b5cbbaa-kube-api-access-ljcwm" (OuterVolumeSpecName: "kube-api-access-ljcwm") pod "806dd485-5977-47d8-b875-fe224b5cbbaa" (UID: "806dd485-5977-47d8-b875-fe224b5cbbaa"). InnerVolumeSpecName "kube-api-access-ljcwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:58:50.538758 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.538720 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182bca5b-f6b3-42c7-b37a-8b58fc795207-kube-api-access-mcf6l" (OuterVolumeSpecName: "kube-api-access-mcf6l") pod "182bca5b-f6b3-42c7-b37a-8b58fc795207" (UID: "182bca5b-f6b3-42c7-b37a-8b58fc795207"). InnerVolumeSpecName "kube-api-access-mcf6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:58:50.557005 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.556979 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:50.565847 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.565826 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:50.637935 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.637899 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljcwm\" (UniqueName: \"kubernetes.io/projected/806dd485-5977-47d8-b875-fe224b5cbbaa-kube-api-access-ljcwm\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:58:50.637935 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.637934 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcf6l\" (UniqueName: \"kubernetes.io/projected/182bca5b-f6b3-42c7-b37a-8b58fc795207-kube-api-access-mcf6l\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:58:50.638156 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.637951 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/806dd485-5977-47d8-b875-fe224b5cbbaa-extensions-socket-volume\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:58:50.729684 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.729657 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb"] Jul 15 13:58:50.731975 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:58:50.731948 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ebfa81_2287_4a58_bf52_efe8660aff90.slice/crio-1751d655f363b22f2667ee985d1b99492abf2210e3b841ba56840e05f66161fe WatchSource:0}: Error finding container 1751d655f363b22f2667ee985d1b99492abf2210e3b841ba56840e05f66161fe: Status 404 returned error can't find the container with id 1751d655f363b22f2667ee985d1b99492abf2210e3b841ba56840e05f66161fe Jul 15 13:58:50.752373 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:50.752354 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc"] Jul 15 13:58:50.754918 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:58:50.754890 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbb351e_8fa5_48b2_bdb7_5de2c3992946.slice/crio-a3a6d94022e28777d1d79841021aff2d5e02c021f0bf2d95e37bce15a85766d8 WatchSource:0}: Error finding container a3a6d94022e28777d1d79841021aff2d5e02c021f0bf2d95e37bce15a85766d8: Status 404 returned error can't find the container with id a3a6d94022e28777d1d79841021aff2d5e02c021f0bf2d95e37bce15a85766d8 Jul 15 13:58:51.219157 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.219119 2584 generic.go:356] "Generic (PLEG): container finished" podID="182bca5b-f6b3-42c7-b37a-8b58fc795207" containerID="2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7" exitCode=0 Jul 15 13:58:51.219599 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.219177 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:51.219599 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.219207 2584 scope.go:117] "RemoveContainer" containerID="2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7" Jul 15 13:58:51.220948 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.220923 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" event={"ID":"dcbb351e-8fa5-48b2-bdb7-5de2c3992946","Type":"ContainerStarted","Data":"1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3"} Jul 15 13:58:51.221041 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.220977 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" event={"ID":"dcbb351e-8fa5-48b2-bdb7-5de2c3992946","Type":"ContainerStarted","Data":"a3a6d94022e28777d1d79841021aff2d5e02c021f0bf2d95e37bce15a85766d8"} Jul 15 13:58:51.221041 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.221023 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:58:51.222031 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.222002 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:51.222763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.222738 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" event={"ID":"e5ebfa81-2287-4a58-bf52-efe8660aff90","Type":"ContainerStarted","Data":"fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1"} Jul 15 13:58:51.222892 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.222782 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" event={"ID":"e5ebfa81-2287-4a58-bf52-efe8660aff90","Type":"ContainerStarted","Data":"1751d655f363b22f2667ee985d1b99492abf2210e3b841ba56840e05f66161fe"} Jul 15 13:58:51.222892 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.222841 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:58:51.224039 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.223979 2584 generic.go:356] "Generic (PLEG): container finished" podID="806dd485-5977-47d8-b875-fe224b5cbbaa" containerID="5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56" exitCode=0 Jul 15 13:58:51.224039 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.224018 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:51.224214 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.224157 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:51.224316 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.224231 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:51.226090 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.226061 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:51.227899 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.227859 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:51.229721 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.229708 2584 scope.go:117] "RemoveContainer" containerID="2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7" Jul 15 13:58:51.229996 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.229980 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7\": container with ID starting with 2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7 not found: ID does not exist" containerID="2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7" Jul 15 13:58:51.230066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.230003 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7"} err="failed to get container status \"2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7\": rpc error: code = NotFound desc = could not find container \"2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7\": container with ID starting with 2a2549f7e43497fc44a7bd648359be08e0e95d86bcd592e11594b4d37a4d64a7 not found: ID does not exist" Jul 15 13:58:51.230066 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.230019 2584 scope.go:117] "RemoveContainer" containerID="5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56" Jul 15 13:58:51.242556 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.242530 2584 scope.go:117] "RemoveContainer" containerID="5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56" Jul 15 13:58:51.242821 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.242804 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56\": container with ID starting with 5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56 not found: ID does not exist" containerID="5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56" Jul 15 13:58:51.242882 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.242830 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56"} err="failed to get container status \"5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56\": rpc error: code = NotFound desc = could not find container \"5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56\": container with ID starting with 5289984d8ebdd232bffd8a90af1db4ace0905ccfe03086b7af7a0e2abf144b56 not found: ID does not exist" Jul 15 13:58:51.252663 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.252627 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" podStartSLOduration=1.2526156560000001 podStartE2EDuration="1.252615656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:51.250475383 +0000 UTC m=+469.688084174" watchObservedRunningTime="2026-07-15 13:58:51.252615656 +0000 UTC m=+469.690224446" Jul 15 13:58:51.252960 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.252939 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:51.277763 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:51.277725 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" podStartSLOduration=1.277712169 podStartE2EDuration="1.277712169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:51.277307427 +0000 UTC m=+469.714916210" watchObservedRunningTime="2026-07-15 13:58:51.277712169 +0000 UTC m=+469.715320959" Jul 15 13:58:51.279430 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.279407 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:51.281453 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.281436 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:58:51.283434 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.283411 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:51.285582 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:51.285562 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:52.007516 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:52.007482 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9khv2\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9khv2" Jul 15 13:58:52.007956 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:52.007932 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" path="/var/lib/kubelet/pods/182bca5b-f6b3-42c7-b37a-8b58fc795207/volumes" Jul 15 13:58:52.008355 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:52.008337 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" path="/var/lib/kubelet/pods/806dd485-5977-47d8-b875-fe224b5cbbaa/volumes" Jul 15 13:58:52.008670 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:58:52.008653 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" path="/var/lib/kubelet/pods/ce35658a-c467-41bc-b228-5963fdd9c484/volumes" Jul 15 13:58:52.013339 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:52.013317 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tnmx8\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="806dd485-5977-47d8-b875-fe224b5cbbaa" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tnmx8" Jul 15 13:58:52.015675 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:58:52.015653 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t58\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ce35658a-c467-41bc-b228-5963fdd9c484" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t58" Jul 15 13:59:02.232252 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.232161 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:59:02.232252 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.232235 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 13:59:02.319743 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.319715 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc"] Jul 15 13:59:02.319925 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.319904 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" podUID="dcbb351e-8fa5-48b2-bdb7-5de2c3992946" containerName="manager" containerID="cri-o://1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3" gracePeriod=10 Jul 15 13:59:02.566870 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.566844 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:59:02.638177 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.638150 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtnxr\" (UniqueName: \"kubernetes.io/projected/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-kube-api-access-gtnxr\") pod \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " Jul 15 13:59:02.638323 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.638241 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-extensions-socket-volume\") pod \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\" (UID: \"dcbb351e-8fa5-48b2-bdb7-5de2c3992946\") " Jul 15 13:59:02.638592 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.638563 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dcbb351e-8fa5-48b2-bdb7-5de2c3992946" (UID: "dcbb351e-8fa5-48b2-bdb7-5de2c3992946"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 13:59:02.640129 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.640112 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-kube-api-access-gtnxr" (OuterVolumeSpecName: "kube-api-access-gtnxr") pod "dcbb351e-8fa5-48b2-bdb7-5de2c3992946" (UID: "dcbb351e-8fa5-48b2-bdb7-5de2c3992946"). InnerVolumeSpecName "kube-api-access-gtnxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:59:02.739462 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.739391 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-extensions-socket-volume\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:59:02.739462 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:02.739416 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtnxr\" (UniqueName: \"kubernetes.io/projected/dcbb351e-8fa5-48b2-bdb7-5de2c3992946-kube-api-access-gtnxr\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:59:03.284106 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.284069 2584 generic.go:356] "Generic (PLEG): container finished" podID="dcbb351e-8fa5-48b2-bdb7-5de2c3992946" containerID="1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3" exitCode=0 Jul 15 13:59:03.284571 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.284121 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" Jul 15 13:59:03.284571 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.284159 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" event={"ID":"dcbb351e-8fa5-48b2-bdb7-5de2c3992946","Type":"ContainerDied","Data":"1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3"} Jul 15 13:59:03.284571 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.284214 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc" event={"ID":"dcbb351e-8fa5-48b2-bdb7-5de2c3992946","Type":"ContainerDied","Data":"a3a6d94022e28777d1d79841021aff2d5e02c021f0bf2d95e37bce15a85766d8"} Jul 15 13:59:03.284571 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.284236 2584 scope.go:117] "RemoveContainer" containerID="1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3" Jul 15 13:59:03.292727 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.292715 2584 scope.go:117] "RemoveContainer" containerID="1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3" Jul 15 13:59:03.293004 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:59:03.292987 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3\": container with ID starting with 1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3 not found: ID does not exist" containerID="1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3" Jul 15 13:59:03.293062 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.293016 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3"} err="failed to get container status \"1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3\": rpc error: code = NotFound desc = could not find container \"1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3\": container with ID starting with 1c109abe263874c9614782667f691afcaf309e1220d81411c8e5c8b0240559c3 not found: ID does not exist" Jul 15 13:59:03.310490 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.310467 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc"] Jul 15 13:59:03.338534 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:03.338504 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kbchc"] Jul 15 13:59:04.007834 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:04.007805 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcbb351e-8fa5-48b2-bdb7-5de2c3992946" path="/var/lib/kubelet/pods/dcbb351e-8fa5-48b2-bdb7-5de2c3992946/volumes" Jul 15 13:59:18.906907 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.906871 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff"] Jul 15 13:59:18.907248 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.907216 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" containerName="manager" Jul 15 13:59:18.907248 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.907227 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" containerName="manager" Jul 15 13:59:18.907248 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.907241 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcbb351e-8fa5-48b2-bdb7-5de2c3992946" containerName="manager" Jul 15 13:59:18.907248 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.907246 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcbb351e-8fa5-48b2-bdb7-5de2c3992946" containerName="manager" Jul 15 13:59:18.907422 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.907325 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcbb351e-8fa5-48b2-bdb7-5de2c3992946" containerName="manager" Jul 15 13:59:18.907422 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.907333 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="182bca5b-f6b3-42c7-b37a-8b58fc795207" containerName="manager" Jul 15 13:59:18.911067 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.911049 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.915766 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.915747 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g8w8d\"" Jul 15 13:59:18.916386 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.916365 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 13:59:18.929582 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.929563 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff"] Jul 15 13:59:18.975717 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975681 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvn75\" (UniqueName: \"kubernetes.io/projected/c637485c-0a7e-45da-9344-046ab1f44771-kube-api-access-gvn75\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.975875 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975729 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.975875 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975798 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.975875 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975823 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.975875 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975845 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c637485c-0a7e-45da-9344-046ab1f44771-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.975875 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.976115 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975887 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c637485c-0a7e-45da-9344-046ab1f44771-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.976115 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.975968 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:18.976115 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:18.976021 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c637485c-0a7e-45da-9344-046ab1f44771-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.076846 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.076810 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077042 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.076876 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077042 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.076902 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077042 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.076931 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c637485c-0a7e-45da-9344-046ab1f44771-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077207 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077072 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077207 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077115 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c637485c-0a7e-45da-9344-046ab1f44771-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077207 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077159 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077395 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c637485c-0a7e-45da-9344-046ab1f44771-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077395 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077209 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077395 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077252 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvn75\" (UniqueName: \"kubernetes.io/projected/c637485c-0a7e-45da-9344-046ab1f44771-kube-api-access-gvn75\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077395 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077253 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077587 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077432 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077587 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077490 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.077841 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.077812 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c637485c-0a7e-45da-9344-046ab1f44771-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.079220 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.079195 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c637485c-0a7e-45da-9344-046ab1f44771-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.079474 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.079457 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c637485c-0a7e-45da-9344-046ab1f44771-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.086553 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.086520 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c637485c-0a7e-45da-9344-046ab1f44771-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.086927 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.086910 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvn75\" (UniqueName: \"kubernetes.io/projected/c637485c-0a7e-45da-9344-046ab1f44771-kube-api-access-gvn75\") pod \"maas-default-gateway-openshift-default-56654dcf7-jgfff\" (UID: \"c637485c-0a7e-45da-9344-046ab1f44771\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.220471 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.220377 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:19.364947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:19.364918 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff"] Jul 15 13:59:19.367083 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:19.367055 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc637485c_0a7e_45da_9344_046ab1f44771.slice/crio-9942293be62ea310eed94a498ff007daa9cd0d5e78e442d2375c00c2d6b75341 WatchSource:0}: Error finding container 9942293be62ea310eed94a498ff007daa9cd0d5e78e442d2375c00c2d6b75341: Status 404 returned error can't find the container with id 9942293be62ea310eed94a498ff007daa9cd0d5e78e442d2375c00c2d6b75341 Jul 15 13:59:20.368139 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:20.368094 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" event={"ID":"c637485c-0a7e-45da-9344-046ab1f44771","Type":"ContainerStarted","Data":"9942293be62ea310eed94a498ff007daa9cd0d5e78e442d2375c00c2d6b75341"} Jul 15 13:59:22.109192 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:22.109157 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 13:59:22.109472 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:22.109229 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 13:59:22.109472 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:22.109257 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 13:59:22.381959 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:22.381867 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" event={"ID":"c637485c-0a7e-45da-9344-046ab1f44771","Type":"ContainerStarted","Data":"29fe2ea3831408831c3c2246072fc520f91cdb2a206e944188b32bb07b2cbaa2"} Jul 15 13:59:22.405989 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:22.405929 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" podStartSLOduration=1.666025182 podStartE2EDuration="4.405913869s" totalImagesPullingTime="2.739888687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:22.403624983 +0000 UTC m=+500.841233773" watchObservedRunningTime="2026-07-15 13:59:22.405913869 +0000 UTC m=+500.843522719" Jul 15 13:59:23.220951 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:23.220915 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:23.225630 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:23.225609 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:23.387007 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:23.386979 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:23.387827 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:23.387805 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 13:59:24.452757 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.452725 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-n5kcz"] Jul 15 13:59:24.457205 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.457186 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:24.460467 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.460444 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dtllk\"" Jul 15 13:59:24.466144 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.466121 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n5kcz"] Jul 15 13:59:24.527252 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.527214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqh2b\" (UniqueName: \"kubernetes.io/projected/bc3d32b5-5156-4816-9f39-31eb36cd015e-kube-api-access-xqh2b\") pod \"authorino-58cd955675-n5kcz\" (UID: \"bc3d32b5-5156-4816-9f39-31eb36cd015e\") " pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:24.586541 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.586505 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:24.591557 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.591529 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.594859 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.594832 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 13:59:24.594972 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.594902 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s2lxs\"" Jul 15 13:59:24.601112 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.601066 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:24.627907 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.627874 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqh2b\" (UniqueName: \"kubernetes.io/projected/bc3d32b5-5156-4816-9f39-31eb36cd015e-kube-api-access-xqh2b\") pod \"authorino-58cd955675-n5kcz\" (UID: \"bc3d32b5-5156-4816-9f39-31eb36cd015e\") " pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:24.645351 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.645322 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqh2b\" (UniqueName: \"kubernetes.io/projected/bc3d32b5-5156-4816-9f39-31eb36cd015e-kube-api-access-xqh2b\") pod \"authorino-58cd955675-n5kcz\" (UID: \"bc3d32b5-5156-4816-9f39-31eb36cd015e\") " pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:24.694510 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.694476 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:24.729170 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.729086 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bef78341-3918-4fe5-ba84-1e0d3e815d90-config-file\") pod \"limitador-limitador-64c94fd64f-8m9pm\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.729170 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.729128 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brfqn\" (UniqueName: \"kubernetes.io/projected/bef78341-3918-4fe5-ba84-1e0d3e815d90-kube-api-access-brfqn\") pod \"limitador-limitador-64c94fd64f-8m9pm\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.765947 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.765911 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:24.830328 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.830258 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bef78341-3918-4fe5-ba84-1e0d3e815d90-config-file\") pod \"limitador-limitador-64c94fd64f-8m9pm\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.830489 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.830354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brfqn\" (UniqueName: \"kubernetes.io/projected/bef78341-3918-4fe5-ba84-1e0d3e815d90-kube-api-access-brfqn\") pod \"limitador-limitador-64c94fd64f-8m9pm\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.831123 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.831092 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bef78341-3918-4fe5-ba84-1e0d3e815d90-config-file\") pod \"limitador-limitador-64c94fd64f-8m9pm\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.839146 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.839119 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brfqn\" (UniqueName: \"kubernetes.io/projected/bef78341-3918-4fe5-ba84-1e0d3e815d90-kube-api-access-brfqn\") pod \"limitador-limitador-64c94fd64f-8m9pm\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.902718 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.902676 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:24.911271 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.911247 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n5kcz"] Jul 15 13:59:24.913754 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:24.913729 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc3d32b5_5156_4816_9f39_31eb36cd015e.slice/crio-06ab6469f51b6d62fb773cbb35156d4ec5e36863e94f32c4023403091e5e9308 WatchSource:0}: Error finding container 06ab6469f51b6d62fb773cbb35156d4ec5e36863e94f32c4023403091e5e9308: Status 404 returned error can't find the container with id 06ab6469f51b6d62fb773cbb35156d4ec5e36863e94f32c4023403091e5e9308 Jul 15 13:59:24.918478 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.918454 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-cb9g7"] Jul 15 13:59:24.923271 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.923248 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 13:59:24.931013 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:24.930940 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cb9g7"] Jul 15 13:59:25.032379 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.032342 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25qhp\" (UniqueName: \"kubernetes.io/projected/a972fc50-e173-49f3-80ae-27b464a6cb46-kube-api-access-25qhp\") pod \"authorino-75d556bd77-cb9g7\" (UID: \"a972fc50-e173-49f3-80ae-27b464a6cb46\") " pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 13:59:25.047457 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.047434 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:25.050178 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:25.050150 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef78341_3918_4fe5_ba84_1e0d3e815d90.slice/crio-7d2e35588637085967b14faf2887a9aced741629bc604d70a21ed52e87f47ac2 WatchSource:0}: Error finding container 7d2e35588637085967b14faf2887a9aced741629bc604d70a21ed52e87f47ac2: Status 404 returned error can't find the container with id 7d2e35588637085967b14faf2887a9aced741629bc604d70a21ed52e87f47ac2 Jul 15 13:59:25.133380 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.133331 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25qhp\" (UniqueName: \"kubernetes.io/projected/a972fc50-e173-49f3-80ae-27b464a6cb46-kube-api-access-25qhp\") pod \"authorino-75d556bd77-cb9g7\" (UID: \"a972fc50-e173-49f3-80ae-27b464a6cb46\") " pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 13:59:25.142797 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.142774 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25qhp\" (UniqueName: \"kubernetes.io/projected/a972fc50-e173-49f3-80ae-27b464a6cb46-kube-api-access-25qhp\") pod \"authorino-75d556bd77-cb9g7\" (UID: \"a972fc50-e173-49f3-80ae-27b464a6cb46\") " pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 13:59:25.234962 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.234929 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 13:59:25.378913 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.378884 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cb9g7"] Jul 15 13:59:25.379767 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:25.379745 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda972fc50_e173_49f3_80ae_27b464a6cb46.slice/crio-5efa608fba081a2e027ed4de16ab91ae0942e873f693635d4626a4d893d45f26 WatchSource:0}: Error finding container 5efa608fba081a2e027ed4de16ab91ae0942e873f693635d4626a4d893d45f26: Status 404 returned error can't find the container with id 5efa608fba081a2e027ed4de16ab91ae0942e873f693635d4626a4d893d45f26 Jul 15 13:59:25.397526 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.397491 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n5kcz" event={"ID":"bc3d32b5-5156-4816-9f39-31eb36cd015e","Type":"ContainerStarted","Data":"06ab6469f51b6d62fb773cbb35156d4ec5e36863e94f32c4023403091e5e9308"} Jul 15 13:59:25.398829 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.398804 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cb9g7" event={"ID":"a972fc50-e173-49f3-80ae-27b464a6cb46","Type":"ContainerStarted","Data":"5efa608fba081a2e027ed4de16ab91ae0942e873f693635d4626a4d893d45f26"} Jul 15 13:59:25.400052 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:25.400031 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" event={"ID":"bef78341-3918-4fe5-ba84-1e0d3e815d90","Type":"ContainerStarted","Data":"7d2e35588637085967b14faf2887a9aced741629bc604d70a21ed52e87f47ac2"} Jul 15 13:59:30.434266 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.434154 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cb9g7" event={"ID":"a972fc50-e173-49f3-80ae-27b464a6cb46","Type":"ContainerStarted","Data":"073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b"} Jul 15 13:59:30.435635 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.435607 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" event={"ID":"bef78341-3918-4fe5-ba84-1e0d3e815d90","Type":"ContainerStarted","Data":"6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5"} Jul 15 13:59:30.435725 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.435699 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:30.436992 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.436973 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n5kcz" event={"ID":"bc3d32b5-5156-4816-9f39-31eb36cd015e","Type":"ContainerStarted","Data":"c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398"} Jul 15 13:59:30.454447 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.454400 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-cb9g7" podStartSLOduration=1.746795513 podStartE2EDuration="6.454385344s" totalImagesPullingTime="4.707589831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:30.452881507 +0000 UTC m=+508.890490298" watchObservedRunningTime="2026-07-15 13:59:30.454385344 +0000 UTC m=+508.891994135" Jul 15 13:59:30.474177 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.474093 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" podStartSLOduration=1.3966149749999999 podStartE2EDuration="6.474072482s" totalImagesPullingTime="5.077457507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:30.471392551 +0000 UTC m=+508.909001342" watchObservedRunningTime="2026-07-15 13:59:30.474072482 +0000 UTC m=+508.911681274" Jul 15 13:59:30.488500 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.488470 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n5kcz"] Jul 15 13:59:30.498893 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:30.498845 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-n5kcz" podStartSLOduration=1.290963147 podStartE2EDuration="6.498831389s" totalImagesPullingTime="5.207868242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:30.498046206 +0000 UTC m=+508.935654995" watchObservedRunningTime="2026-07-15 13:59:30.498831389 +0000 UTC m=+508.936440183" Jul 15 13:59:32.446162 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:32.446121 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-n5kcz" podUID="bc3d32b5-5156-4816-9f39-31eb36cd015e" containerName="authorino" containerID="cri-o://c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398" gracePeriod=30 Jul 15 13:59:32.702498 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:32.702437 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:32.807477 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:32.807435 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqh2b\" (UniqueName: \"kubernetes.io/projected/bc3d32b5-5156-4816-9f39-31eb36cd015e-kube-api-access-xqh2b\") pod \"bc3d32b5-5156-4816-9f39-31eb36cd015e\" (UID: \"bc3d32b5-5156-4816-9f39-31eb36cd015e\") " Jul 15 13:59:32.809445 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:32.809413 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc3d32b5-5156-4816-9f39-31eb36cd015e-kube-api-access-xqh2b" (OuterVolumeSpecName: "kube-api-access-xqh2b") pod "bc3d32b5-5156-4816-9f39-31eb36cd015e" (UID: "bc3d32b5-5156-4816-9f39-31eb36cd015e"). InnerVolumeSpecName "kube-api-access-xqh2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:59:32.908857 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:32.908815 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqh2b\" (UniqueName: \"kubernetes.io/projected/bc3d32b5-5156-4816-9f39-31eb36cd015e-kube-api-access-xqh2b\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:59:33.452433 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.452401 2584 generic.go:356] "Generic (PLEG): container finished" podID="bc3d32b5-5156-4816-9f39-31eb36cd015e" containerID="c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398" exitCode=0 Jul 15 13:59:33.452433 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.452436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n5kcz" event={"ID":"bc3d32b5-5156-4816-9f39-31eb36cd015e","Type":"ContainerDied","Data":"c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398"} Jul 15 13:59:33.452928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.452442 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n5kcz" Jul 15 13:59:33.452928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.452462 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n5kcz" event={"ID":"bc3d32b5-5156-4816-9f39-31eb36cd015e","Type":"ContainerDied","Data":"06ab6469f51b6d62fb773cbb35156d4ec5e36863e94f32c4023403091e5e9308"} Jul 15 13:59:33.452928 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.452478 2584 scope.go:117] "RemoveContainer" containerID="c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398" Jul 15 13:59:33.466910 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.466885 2584 scope.go:117] "RemoveContainer" containerID="c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398" Jul 15 13:59:33.467176 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:59:33.467157 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398\": container with ID starting with c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398 not found: ID does not exist" containerID="c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398" Jul 15 13:59:33.467218 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.467185 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398"} err="failed to get container status \"c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398\": rpc error: code = NotFound desc = could not find container \"c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398\": container with ID starting with c1b6d67dce4493baacf2001e889dd47b5d8bcb87f63b743a21e10554fe3f5398 not found: ID does not exist" Jul 15 13:59:33.478478 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.478453 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n5kcz"] Jul 15 13:59:33.482611 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:33.482587 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-n5kcz"] Jul 15 13:59:34.008044 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:34.008008 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc3d32b5-5156-4816-9f39-31eb36cd015e" path="/var/lib/kubelet/pods/bc3d32b5-5156-4816-9f39-31eb36cd015e/volumes" Jul 15 13:59:39.823049 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:39.823013 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:39.823479 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:39.823243 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" podUID="bef78341-3918-4fe5-ba84-1e0d3e815d90" containerName="limitador" containerID="cri-o://6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5" gracePeriod=30 Jul 15 13:59:39.825253 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:39.825230 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:40.371234 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.371209 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:40.469625 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.469585 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brfqn\" (UniqueName: \"kubernetes.io/projected/bef78341-3918-4fe5-ba84-1e0d3e815d90-kube-api-access-brfqn\") pod \"bef78341-3918-4fe5-ba84-1e0d3e815d90\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " Jul 15 13:59:40.469803 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.469698 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bef78341-3918-4fe5-ba84-1e0d3e815d90-config-file\") pod \"bef78341-3918-4fe5-ba84-1e0d3e815d90\" (UID: \"bef78341-3918-4fe5-ba84-1e0d3e815d90\") " Jul 15 13:59:40.470041 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.470011 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef78341-3918-4fe5-ba84-1e0d3e815d90-config-file" (OuterVolumeSpecName: "config-file") pod "bef78341-3918-4fe5-ba84-1e0d3e815d90" (UID: "bef78341-3918-4fe5-ba84-1e0d3e815d90"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 13:59:40.471706 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.471683 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bef78341-3918-4fe5-ba84-1e0d3e815d90-kube-api-access-brfqn" (OuterVolumeSpecName: "kube-api-access-brfqn") pod "bef78341-3918-4fe5-ba84-1e0d3e815d90" (UID: "bef78341-3918-4fe5-ba84-1e0d3e815d90"). InnerVolumeSpecName "kube-api-access-brfqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 13:59:40.488755 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.488724 2584 generic.go:356] "Generic (PLEG): container finished" podID="bef78341-3918-4fe5-ba84-1e0d3e815d90" containerID="6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5" exitCode=0 Jul 15 13:59:40.488929 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.488763 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" Jul 15 13:59:40.488929 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.488767 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" event={"ID":"bef78341-3918-4fe5-ba84-1e0d3e815d90","Type":"ContainerDied","Data":"6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5"} Jul 15 13:59:40.488929 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.488868 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8m9pm" event={"ID":"bef78341-3918-4fe5-ba84-1e0d3e815d90","Type":"ContainerDied","Data":"7d2e35588637085967b14faf2887a9aced741629bc604d70a21ed52e87f47ac2"} Jul 15 13:59:40.488929 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.488891 2584 scope.go:117] "RemoveContainer" containerID="6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5" Jul 15 13:59:40.497753 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.497732 2584 scope.go:117] "RemoveContainer" containerID="6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5" Jul 15 13:59:40.498052 ip-10-0-133-21 kubenswrapper[2584]: E0715 13:59:40.498032 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5\": container with ID starting with 6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5 not found: ID does not exist" containerID="6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5" Jul 15 13:59:40.498106 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.498062 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5"} err="failed to get container status \"6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5\": rpc error: code = NotFound desc = could not find container \"6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5\": container with ID starting with 6880ea05bf232a4a32a9a4439170c3eefee602b95741db7859ea015aea6973e5 not found: ID does not exist" Jul 15 13:59:40.511078 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.511045 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:40.515085 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.515062 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8m9pm"] Jul 15 13:59:40.571101 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.571066 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brfqn\" (UniqueName: \"kubernetes.io/projected/bef78341-3918-4fe5-ba84-1e0d3e815d90-kube-api-access-brfqn\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:59:40.571101 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:40.571097 2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bef78341-3918-4fe5-ba84-1e0d3e815d90-config-file\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 13:59:42.008863 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:42.008833 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bef78341-3918-4fe5-ba84-1e0d3e815d90" path="/var/lib/kubelet/pods/bef78341-3918-4fe5-ba84-1e0d3e815d90/volumes" Jul 15 13:59:45.432224 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432188 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz"] Jul 15 13:59:45.432588 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432556 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc3d32b5-5156-4816-9f39-31eb36cd015e" containerName="authorino" Jul 15 13:59:45.432588 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432568 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc3d32b5-5156-4816-9f39-31eb36cd015e" containerName="authorino" Jul 15 13:59:45.432671 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432590 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bef78341-3918-4fe5-ba84-1e0d3e815d90" containerName="limitador" Jul 15 13:59:45.432671 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432596 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef78341-3918-4fe5-ba84-1e0d3e815d90" containerName="limitador" Jul 15 13:59:45.432671 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432652 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc3d32b5-5156-4816-9f39-31eb36cd015e" containerName="authorino" Jul 15 13:59:45.432671 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.432665 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bef78341-3918-4fe5-ba84-1e0d3e815d90" containerName="limitador" Jul 15 13:59:45.436764 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.436743 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.439257 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.439235 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 15 13:59:45.439397 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.439242 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 13:59:45.440338 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.440320 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8v7ks\"" Jul 15 13:59:45.440393 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.440351 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 13:59:45.448906 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.448887 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz"] Jul 15 13:59:45.617229 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.617192 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3f6bf743-bcef-4eeb-b3f0-516365bc09d7-data\") pod \"postgres-69fb4594cf-ndsfz\" (UID: \"3f6bf743-bcef-4eeb-b3f0-516365bc09d7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.617411 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.617249 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25jtf\" (UniqueName: \"kubernetes.io/projected/3f6bf743-bcef-4eeb-b3f0-516365bc09d7-kube-api-access-25jtf\") pod \"postgres-69fb4594cf-ndsfz\" (UID: \"3f6bf743-bcef-4eeb-b3f0-516365bc09d7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.718718 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.718645 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25jtf\" (UniqueName: \"kubernetes.io/projected/3f6bf743-bcef-4eeb-b3f0-516365bc09d7-kube-api-access-25jtf\") pod \"postgres-69fb4594cf-ndsfz\" (UID: \"3f6bf743-bcef-4eeb-b3f0-516365bc09d7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.718840 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.718729 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3f6bf743-bcef-4eeb-b3f0-516365bc09d7-data\") pod \"postgres-69fb4594cf-ndsfz\" (UID: \"3f6bf743-bcef-4eeb-b3f0-516365bc09d7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.719028 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.719013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3f6bf743-bcef-4eeb-b3f0-516365bc09d7-data\") pod \"postgres-69fb4594cf-ndsfz\" (UID: \"3f6bf743-bcef-4eeb-b3f0-516365bc09d7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.728343 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.728318 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25jtf\" (UniqueName: \"kubernetes.io/projected/3f6bf743-bcef-4eeb-b3f0-516365bc09d7-kube-api-access-25jtf\") pod \"postgres-69fb4594cf-ndsfz\" (UID: \"3f6bf743-bcef-4eeb-b3f0-516365bc09d7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.746258 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.746236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:45.882598 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:45.882572 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz"] Jul 15 13:59:45.884647 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:45.884623 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6bf743_bcef_4eeb_b3f0_516365bc09d7.slice/crio-9e0fb30f6e07c9b53d7975f99f7981ae4b880e912fa0c5790cb8a49fecb31baf WatchSource:0}: Error finding container 9e0fb30f6e07c9b53d7975f99f7981ae4b880e912fa0c5790cb8a49fecb31baf: Status 404 returned error can't find the container with id 9e0fb30f6e07c9b53d7975f99f7981ae4b880e912fa0c5790cb8a49fecb31baf Jul 15 13:59:46.518444 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:46.518410 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" event={"ID":"3f6bf743-bcef-4eeb-b3f0-516365bc09d7","Type":"ContainerStarted","Data":"9e0fb30f6e07c9b53d7975f99f7981ae4b880e912fa0c5790cb8a49fecb31baf"} Jul 15 13:59:51.549466 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:51.549417 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" event={"ID":"3f6bf743-bcef-4eeb-b3f0-516365bc09d7","Type":"ContainerStarted","Data":"a9f8804f679d7b6ab4ec8f9329b683bca2a48ba42f79bd489e75607d3f94ba37"} Jul 15 13:59:51.549824 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:51.549523 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:51.569246 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:51.569191 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" podStartSLOduration=1.460716235 podStartE2EDuration="6.56917655s" totalImagesPullingTime="5.108460315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:51.567430594 +0000 UTC m=+530.005039390" watchObservedRunningTime="2026-07-15 13:59:51.56917655 +0000 UTC m=+530.006785342" Jul 15 13:59:57.583789 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:57.583712 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ndsfz" Jul 15 13:59:58.215406 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.215373 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dhgb5"] Jul 15 13:59:58.218970 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.218952 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 13:59:58.228275 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.228248 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dhgb5"] Jul 15 13:59:58.333450 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.333415 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fn2v\" (UniqueName: \"kubernetes.io/projected/9a55effa-dc61-4f90-ba19-9698910e8ecb-kube-api-access-2fn2v\") pod \"authorino-5d4b67c446-dhgb5\" (UID: \"9a55effa-dc61-4f90-ba19-9698910e8ecb\") " pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 13:59:58.434584 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.434551 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fn2v\" (UniqueName: \"kubernetes.io/projected/9a55effa-dc61-4f90-ba19-9698910e8ecb-kube-api-access-2fn2v\") pod \"authorino-5d4b67c446-dhgb5\" (UID: \"9a55effa-dc61-4f90-ba19-9698910e8ecb\") " pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 13:59:58.444600 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.444571 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fn2v\" (UniqueName: \"kubernetes.io/projected/9a55effa-dc61-4f90-ba19-9698910e8ecb-kube-api-access-2fn2v\") pod \"authorino-5d4b67c446-dhgb5\" (UID: \"9a55effa-dc61-4f90-ba19-9698910e8ecb\") " pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 13:59:58.517527 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.517436 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dhgb5"] Jul 15 13:59:58.517700 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.517688 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 13:59:58.559573 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.559542 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-594b9d8797-4glwk"] Jul 15 13:59:58.563049 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.563032 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.568021 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.567882 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 13:59:58.581421 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.581367 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594b9d8797-4glwk"] Jul 15 13:59:58.636517 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.636487 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b7527c6-80bf-4af4-bc14-9d94e8682217-tls-cert\") pod \"authorino-594b9d8797-4glwk\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.637050 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.636602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkvxq\" (UniqueName: \"kubernetes.io/projected/0b7527c6-80bf-4af4-bc14-9d94e8682217-kube-api-access-jkvxq\") pod \"authorino-594b9d8797-4glwk\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.657428 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.657406 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dhgb5"] Jul 15 13:59:58.659733 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:58.659695 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a55effa_dc61_4f90_ba19_9698910e8ecb.slice/crio-b7f1185bdea6af7ca2f1aa98edaa234f29aebe1fe8573c89cdb44c60a6ca6440 WatchSource:0}: Error finding container b7f1185bdea6af7ca2f1aa98edaa234f29aebe1fe8573c89cdb44c60a6ca6440: Status 404 returned error can't find the container with id b7f1185bdea6af7ca2f1aa98edaa234f29aebe1fe8573c89cdb44c60a6ca6440 Jul 15 13:59:58.737151 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.737119 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b7527c6-80bf-4af4-bc14-9d94e8682217-tls-cert\") pod \"authorino-594b9d8797-4glwk\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.737366 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.737227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkvxq\" (UniqueName: \"kubernetes.io/projected/0b7527c6-80bf-4af4-bc14-9d94e8682217-kube-api-access-jkvxq\") pod \"authorino-594b9d8797-4glwk\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.739625 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.739603 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b7527c6-80bf-4af4-bc14-9d94e8682217-tls-cert\") pod \"authorino-594b9d8797-4glwk\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.746649 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.746616 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkvxq\" (UniqueName: \"kubernetes.io/projected/0b7527c6-80bf-4af4-bc14-9d94e8682217-kube-api-access-jkvxq\") pod \"authorino-594b9d8797-4glwk\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:58.875044 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:58.875007 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 13:59:59.007518 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.007496 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594b9d8797-4glwk"] Jul 15 13:59:59.009692 ip-10-0-133-21 kubenswrapper[2584]: W0715 13:59:59.009667 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b7527c6_80bf_4af4_bc14_9d94e8682217.slice/crio-723b969c1bd28249e7c6476b862474a99bfc57135b470edb083625fe9d29b5c2 WatchSource:0}: Error finding container 723b969c1bd28249e7c6476b862474a99bfc57135b470edb083625fe9d29b5c2: Status 404 returned error can't find the container with id 723b969c1bd28249e7c6476b862474a99bfc57135b470edb083625fe9d29b5c2 Jul 15 13:59:59.592029 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.591990 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594b9d8797-4glwk" event={"ID":"0b7527c6-80bf-4af4-bc14-9d94e8682217","Type":"ContainerStarted","Data":"282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d"} Jul 15 13:59:59.592233 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.592040 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594b9d8797-4glwk" event={"ID":"0b7527c6-80bf-4af4-bc14-9d94e8682217","Type":"ContainerStarted","Data":"723b969c1bd28249e7c6476b862474a99bfc57135b470edb083625fe9d29b5c2"} Jul 15 13:59:59.593481 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.593459 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" event={"ID":"9a55effa-dc61-4f90-ba19-9698910e8ecb","Type":"ContainerStarted","Data":"6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136"} Jul 15 13:59:59.593592 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.593490 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" event={"ID":"9a55effa-dc61-4f90-ba19-9698910e8ecb","Type":"ContainerStarted","Data":"b7f1185bdea6af7ca2f1aa98edaa234f29aebe1fe8573c89cdb44c60a6ca6440"} Jul 15 13:59:59.593592 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.593519 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" podUID="9a55effa-dc61-4f90-ba19-9698910e8ecb" containerName="authorino" containerID="cri-o://6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136" gracePeriod=30 Jul 15 13:59:59.624590 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.624543 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-594b9d8797-4glwk" podStartSLOduration=1.266580351 podStartE2EDuration="1.624526316s" totalImagesPullingTime="357.945965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:59.623063968 +0000 UTC m=+538.060672760" watchObservedRunningTime="2026-07-15 13:59:59.624526316 +0000 UTC m=+538.062135107" Jul 15 13:59:59.649933 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.649886 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" podStartSLOduration=1.202166505 podStartE2EDuration="1.649866009s" totalImagesPullingTime="447.699504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:59.647280322 +0000 UTC m=+538.084889116" watchObservedRunningTime="2026-07-15 13:59:59.649866009 +0000 UTC m=+538.087474800" Jul 15 13:59:59.668317 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.668223 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cb9g7"] Jul 15 13:59:59.668434 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.668408 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-cb9g7" podUID="a972fc50-e173-49f3-80ae-27b464a6cb46" containerName="authorino" containerID="cri-o://073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b" gracePeriod=30 Jul 15 13:59:59.945154 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.945132 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 13:59:59.947981 ip-10-0-133-21 kubenswrapper[2584]: I0715 13:59:59.947957 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 14:00:00.049876 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.049844 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fn2v\" (UniqueName: \"kubernetes.io/projected/9a55effa-dc61-4f90-ba19-9698910e8ecb-kube-api-access-2fn2v\") pod \"9a55effa-dc61-4f90-ba19-9698910e8ecb\" (UID: \"9a55effa-dc61-4f90-ba19-9698910e8ecb\") " Jul 15 14:00:00.050037 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.049899 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25qhp\" (UniqueName: \"kubernetes.io/projected/a972fc50-e173-49f3-80ae-27b464a6cb46-kube-api-access-25qhp\") pod \"a972fc50-e173-49f3-80ae-27b464a6cb46\" (UID: \"a972fc50-e173-49f3-80ae-27b464a6cb46\") " Jul 15 14:00:00.051900 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.051876 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a55effa-dc61-4f90-ba19-9698910e8ecb-kube-api-access-2fn2v" (OuterVolumeSpecName: "kube-api-access-2fn2v") pod "9a55effa-dc61-4f90-ba19-9698910e8ecb" (UID: "9a55effa-dc61-4f90-ba19-9698910e8ecb"). InnerVolumeSpecName "kube-api-access-2fn2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:00:00.051995 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.051917 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a972fc50-e173-49f3-80ae-27b464a6cb46-kube-api-access-25qhp" (OuterVolumeSpecName: "kube-api-access-25qhp") pod "a972fc50-e173-49f3-80ae-27b464a6cb46" (UID: "a972fc50-e173-49f3-80ae-27b464a6cb46"). InnerVolumeSpecName "kube-api-access-25qhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:00:00.151199 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.151167 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fn2v\" (UniqueName: \"kubernetes.io/projected/9a55effa-dc61-4f90-ba19-9698910e8ecb-kube-api-access-2fn2v\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:00:00.151199 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.151195 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25qhp\" (UniqueName: \"kubernetes.io/projected/a972fc50-e173-49f3-80ae-27b464a6cb46-kube-api-access-25qhp\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:00:00.599963 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.599932 2584 generic.go:356] "Generic (PLEG): container finished" podID="9a55effa-dc61-4f90-ba19-9698910e8ecb" containerID="6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136" exitCode=0 Jul 15 14:00:00.600149 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.599976 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" Jul 15 14:00:00.600149 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.600022 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" event={"ID":"9a55effa-dc61-4f90-ba19-9698910e8ecb","Type":"ContainerDied","Data":"6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136"} Jul 15 14:00:00.600149 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.600060 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dhgb5" event={"ID":"9a55effa-dc61-4f90-ba19-9698910e8ecb","Type":"ContainerDied","Data":"b7f1185bdea6af7ca2f1aa98edaa234f29aebe1fe8573c89cdb44c60a6ca6440"} Jul 15 14:00:00.600149 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.600078 2584 scope.go:117] "RemoveContainer" containerID="6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136" Jul 15 14:00:00.601585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.601566 2584 generic.go:356] "Generic (PLEG): container finished" podID="a972fc50-e173-49f3-80ae-27b464a6cb46" containerID="073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b" exitCode=0 Jul 15 14:00:00.601708 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.601628 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cb9g7" Jul 15 14:00:00.601708 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.601654 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cb9g7" event={"ID":"a972fc50-e173-49f3-80ae-27b464a6cb46","Type":"ContainerDied","Data":"073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b"} Jul 15 14:00:00.601708 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.601696 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cb9g7" event={"ID":"a972fc50-e173-49f3-80ae-27b464a6cb46","Type":"ContainerDied","Data":"5efa608fba081a2e027ed4de16ab91ae0942e873f693635d4626a4d893d45f26"} Jul 15 14:00:00.610007 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.609986 2584 scope.go:117] "RemoveContainer" containerID="6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136" Jul 15 14:00:00.610277 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:00.610246 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136\": container with ID starting with 6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136 not found: ID does not exist" containerID="6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136" Jul 15 14:00:00.610384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.610273 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136"} err="failed to get container status \"6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136\": rpc error: code = NotFound desc = could not find container \"6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136\": container with ID starting with 6aa2ec2e3d7b07c8cbff82dcf59832c6ea9d0f04206ed6d4adfbce874d9b7136 not found: ID does not exist" Jul 15 14:00:00.610384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.610310 2584 scope.go:117] "RemoveContainer" containerID="073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b" Jul 15 14:00:00.617981 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.617965 2584 scope.go:117] "RemoveContainer" containerID="073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b" Jul 15 14:00:00.618225 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:00.618208 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b\": container with ID starting with 073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b not found: ID does not exist" containerID="073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b" Jul 15 14:00:00.618284 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.618235 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b"} err="failed to get container status \"073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b\": rpc error: code = NotFound desc = could not find container \"073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b\": container with ID starting with 073a2bc1c26997f5c1a3e09456949cc225a18ec2ee2844275ef3d221219f086b not found: ID does not exist" Jul 15 14:00:00.633382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.633358 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cb9g7"] Jul 15 14:00:00.636125 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.636099 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cb9g7"] Jul 15 14:00:00.656809 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.656788 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dhgb5"] Jul 15 14:00:00.661418 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:00.661397 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dhgb5"] Jul 15 14:00:02.007155 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:02.007129 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a55effa-dc61-4f90-ba19-9698910e8ecb" path="/var/lib/kubelet/pods/9a55effa-dc61-4f90-ba19-9698910e8ecb/volumes" Jul 15 14:00:02.007497 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:02.007477 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a972fc50-e173-49f3-80ae-27b464a6cb46" path="/var/lib/kubelet/pods/a972fc50-e173-49f3-80ae-27b464a6cb46/volumes" Jul 15 14:00:04.770392 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770359 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68758cc674-s2j6k"] Jul 15 14:00:04.770770 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770713 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a55effa-dc61-4f90-ba19-9698910e8ecb" containerName="authorino" Jul 15 14:00:04.770770 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770725 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a55effa-dc61-4f90-ba19-9698910e8ecb" containerName="authorino" Jul 15 14:00:04.770770 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770740 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a972fc50-e173-49f3-80ae-27b464a6cb46" containerName="authorino" Jul 15 14:00:04.770770 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770745 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a972fc50-e173-49f3-80ae-27b464a6cb46" containerName="authorino" Jul 15 14:00:04.770889 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770810 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a55effa-dc61-4f90-ba19-9698910e8ecb" containerName="authorino" Jul 15 14:00:04.770889 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.770820 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a972fc50-e173-49f3-80ae-27b464a6cb46" containerName="authorino" Jul 15 14:00:04.774947 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.774854 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:04.777714 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.777693 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 14:00:04.777833 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.777726 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m6j56\"" Jul 15 14:00:04.777833 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.777753 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 14:00:04.785840 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.785821 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68758cc674-s2j6k"] Jul 15 14:00:04.888826 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.888793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbbjk\" (UniqueName: \"kubernetes.io/projected/1339c1f9-7e3c-49f5-a443-c05ca140fc65-kube-api-access-xbbjk\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:04.888995 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.888918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:04.968314 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.968273 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-688d755c47-6xlgm"] Jul 15 14:00:04.971481 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.971465 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:04.985492 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.985465 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-688d755c47-6xlgm"] Jul 15 14:00:04.989512 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.989489 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:04.989612 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:04.989538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbbjk\" (UniqueName: \"kubernetes.io/projected/1339c1f9-7e3c-49f5-a443-c05ca140fc65-kube-api-access-xbbjk\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:04.989664 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:04.989648 2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 15 14:00:04.989713 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:04.989704 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert podName:1339c1f9-7e3c-49f5-a443-c05ca140fc65 nodeName:}" failed. No retries permitted until 2026-07-15 14:00:05.489688925 +0000 UTC m=+543.927297695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert") pod "maas-controller-68758cc674-s2j6k" (UID: "1339c1f9-7e3c-49f5-a443-c05ca140fc65") : secret "maas-controller-webhook-cert" not found Jul 15 14:00:05.001701 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.001665 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbbjk\" (UniqueName: \"kubernetes.io/projected/1339c1f9-7e3c-49f5-a443-c05ca140fc65-kube-api-access-xbbjk\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:05.090956 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.090918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.091112 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.090973 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4smcj\" (UniqueName: \"kubernetes.io/projected/8930f726-be31-4031-a3fc-961049b1d892-kube-api-access-4smcj\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.191758 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.191719 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.191758 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.191762 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4smcj\" (UniqueName: \"kubernetes.io/projected/8930f726-be31-4031-a3fc-961049b1d892-kube-api-access-4smcj\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.191963 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:05.191881 2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 15 14:00:05.191963 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:05.191956 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert podName:8930f726-be31-4031-a3fc-961049b1d892 nodeName:}" failed. No retries permitted until 2026-07-15 14:00:05.69193688 +0000 UTC m=+544.129545654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert") pod "maas-controller-688d755c47-6xlgm" (UID: "8930f726-be31-4031-a3fc-961049b1d892") : secret "maas-controller-webhook-cert" not found Jul 15 14:00:05.201153 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.201127 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4smcj\" (UniqueName: \"kubernetes.io/projected/8930f726-be31-4031-a3fc-961049b1d892-kube-api-access-4smcj\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.494450 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.494361 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:05.496831 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.496794 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert\") pod \"maas-controller-68758cc674-s2j6k\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:05.685792 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.685753 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:05.696585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.696558 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.699165 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.699137 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert\") pod \"maas-controller-688d755c47-6xlgm\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:05.823906 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.823878 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68758cc674-s2j6k"] Jul 15 14:00:05.826115 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:00:05.826089 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1339c1f9_7e3c_49f5_a443_c05ca140fc65.slice/crio-f30789eed76bb9ee54ff2db7a0f754fc22d155df6fea6db18194707a6835789d WatchSource:0}: Error finding container f30789eed76bb9ee54ff2db7a0f754fc22d155df6fea6db18194707a6835789d: Status 404 returned error can't find the container with id f30789eed76bb9ee54ff2db7a0f754fc22d155df6fea6db18194707a6835789d Jul 15 14:00:05.881918 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:05.881871 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:06.016560 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:06.016534 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-688d755c47-6xlgm"] Jul 15 14:00:06.017958 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:00:06.017924 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8930f726_be31_4031_a3fc_961049b1d892.slice/crio-57d36b9a2980b35c0a99733faeaac3f9f16f596c437beeeb8745c42085e5eb9e WatchSource:0}: Error finding container 57d36b9a2980b35c0a99733faeaac3f9f16f596c437beeeb8745c42085e5eb9e: Status 404 returned error can't find the container with id 57d36b9a2980b35c0a99733faeaac3f9f16f596c437beeeb8745c42085e5eb9e Jul 15 14:00:06.640000 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:06.639963 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68758cc674-s2j6k" event={"ID":"1339c1f9-7e3c-49f5-a443-c05ca140fc65","Type":"ContainerStarted","Data":"f30789eed76bb9ee54ff2db7a0f754fc22d155df6fea6db18194707a6835789d"} Jul 15 14:00:06.641466 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:06.641436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688d755c47-6xlgm" event={"ID":"8930f726-be31-4031-a3fc-961049b1d892","Type":"ContainerStarted","Data":"57d36b9a2980b35c0a99733faeaac3f9f16f596c437beeeb8745c42085e5eb9e"} Jul 15 14:00:09.661515 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:09.661471 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68758cc674-s2j6k" event={"ID":"1339c1f9-7e3c-49f5-a443-c05ca140fc65","Type":"ContainerStarted","Data":"a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718"} Jul 15 14:00:09.661993 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:09.661551 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:09.663201 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:09.663179 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688d755c47-6xlgm" event={"ID":"8930f726-be31-4031-a3fc-961049b1d892","Type":"ContainerStarted","Data":"7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6"} Jul 15 14:00:09.663339 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:09.663327 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:09.718569 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:09.718510 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68758cc674-s2j6k" podStartSLOduration=2.809936608 podStartE2EDuration="5.718494321s" totalImagesPullingTime="2.908557713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:00:09.718403958 +0000 UTC m=+548.156012753" watchObservedRunningTime="2026-07-15 14:00:09.718494321 +0000 UTC m=+548.156103113" Jul 15 14:00:09.763151 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:09.763101 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-688d755c47-6xlgm" podStartSLOduration=3.05747168 podStartE2EDuration="5.763085283s" totalImagesPullingTime="2.705613603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:00:09.762191343 +0000 UTC m=+548.199800138" watchObservedRunningTime="2026-07-15 14:00:09.763085283 +0000 UTC m=+548.200694076" Jul 15 14:00:20.671355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:20.671324 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:20.671720 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:20.671394 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:00:20.739900 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:20.739872 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68758cc674-s2j6k"] Jul 15 14:00:20.740056 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:20.740025 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68758cc674-s2j6k" podUID="1339c1f9-7e3c-49f5-a443-c05ca140fc65" containerName="manager" containerID="cri-o://a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718" gracePeriod=30 Jul 15 14:00:20.986522 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:20.986495 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:21.134804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.134769 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbbjk\" (UniqueName: \"kubernetes.io/projected/1339c1f9-7e3c-49f5-a443-c05ca140fc65-kube-api-access-xbbjk\") pod \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " Jul 15 14:00:21.135022 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.134892 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert\") pod \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\" (UID: \"1339c1f9-7e3c-49f5-a443-c05ca140fc65\") " Jul 15 14:00:21.136881 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.136855 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1339c1f9-7e3c-49f5-a443-c05ca140fc65" (UID: "1339c1f9-7e3c-49f5-a443-c05ca140fc65"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 14:00:21.136960 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.136897 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1339c1f9-7e3c-49f5-a443-c05ca140fc65-kube-api-access-xbbjk" (OuterVolumeSpecName: "kube-api-access-xbbjk") pod "1339c1f9-7e3c-49f5-a443-c05ca140fc65" (UID: "1339c1f9-7e3c-49f5-a443-c05ca140fc65"). InnerVolumeSpecName "kube-api-access-xbbjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:00:21.235598 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.235520 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1339c1f9-7e3c-49f5-a443-c05ca140fc65-webhook-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:00:21.235598 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.235548 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbbjk\" (UniqueName: \"kubernetes.io/projected/1339c1f9-7e3c-49f5-a443-c05ca140fc65-kube-api-access-xbbjk\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:00:21.722083 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.722051 2584 generic.go:356] "Generic (PLEG): container finished" podID="1339c1f9-7e3c-49f5-a443-c05ca140fc65" containerID="a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718" exitCode=0 Jul 15 14:00:21.722513 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.722095 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68758cc674-s2j6k" Jul 15 14:00:21.722513 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.722116 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68758cc674-s2j6k" event={"ID":"1339c1f9-7e3c-49f5-a443-c05ca140fc65","Type":"ContainerDied","Data":"a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718"} Jul 15 14:00:21.722513 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.722156 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68758cc674-s2j6k" event={"ID":"1339c1f9-7e3c-49f5-a443-c05ca140fc65","Type":"ContainerDied","Data":"f30789eed76bb9ee54ff2db7a0f754fc22d155df6fea6db18194707a6835789d"} Jul 15 14:00:21.722513 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.722178 2584 scope.go:117] "RemoveContainer" containerID="a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718" Jul 15 14:00:21.731470 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.731452 2584 scope.go:117] "RemoveContainer" containerID="a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718" Jul 15 14:00:21.731705 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:00:21.731689 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718\": container with ID starting with a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718 not found: ID does not exist" containerID="a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718" Jul 15 14:00:21.731768 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.731712 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718"} err="failed to get container status \"a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718\": rpc error: code = NotFound desc = could not find container \"a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718\": container with ID starting with a6bf4f8f5c32f5dcec640078482fd82aa9b0de5632cc03d1020f3b908f403718 not found: ID does not exist" Jul 15 14:00:21.744716 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.744692 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68758cc674-s2j6k"] Jul 15 14:00:21.749895 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:21.749872 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68758cc674-s2j6k"] Jul 15 14:00:22.007624 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:00:22.007540 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1339c1f9-7e3c-49f5-a443-c05ca140fc65" path="/var/lib/kubelet/pods/1339c1f9-7e3c-49f5-a443-c05ca140fc65/volumes" Jul 15 14:01:01.944266 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:01.944237 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:01:01.946470 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:01.946442 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:01:01.952550 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:01.952522 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:01:01.954863 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:01.954843 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:01:12.541446 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.541401 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl"] Jul 15 14:01:12.542003 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.541973 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1339c1f9-7e3c-49f5-a443-c05ca140fc65" containerName="manager" Jul 15 14:01:12.542003 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.541992 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1339c1f9-7e3c-49f5-a443-c05ca140fc65" containerName="manager" Jul 15 14:01:12.542120 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.542071 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1339c1f9-7e3c-49f5-a443-c05ca140fc65" containerName="manager" Jul 15 14:01:12.546273 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.546250 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.550166 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.550143 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 15 14:01:12.550308 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.550281 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hr6tq\"" Jul 15 14:01:12.565512 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.565490 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl"] Jul 15 14:01:12.674314 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.674265 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2q79\" (UniqueName: \"kubernetes.io/projected/19947c84-8671-40d9-8124-26148f6491fb-kube-api-access-x2q79\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.674490 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.674334 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19947c84-8671-40d9-8124-26148f6491fb-maas-api-tls\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.674490 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.674395 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19947c84-8671-40d9-8124-26148f6491fb-openshift-service-ca\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.775506 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.775478 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19947c84-8671-40d9-8124-26148f6491fb-openshift-service-ca\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.775665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.775535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2q79\" (UniqueName: \"kubernetes.io/projected/19947c84-8671-40d9-8124-26148f6491fb-kube-api-access-x2q79\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.775665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.775562 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19947c84-8671-40d9-8124-26148f6491fb-maas-api-tls\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.776148 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.776129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19947c84-8671-40d9-8124-26148f6491fb-openshift-service-ca\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.778006 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.777980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19947c84-8671-40d9-8124-26148f6491fb-maas-api-tls\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.790214 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.790189 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2q79\" (UniqueName: \"kubernetes.io/projected/19947c84-8671-40d9-8124-26148f6491fb-kube-api-access-x2q79\") pod \"maas-api-d75896c4d-r7dbl\" (UID: \"19947c84-8671-40d9-8124-26148f6491fb\") " pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.857040 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.857006 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:12.995689 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:12.995658 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl"] Jul 15 14:01:12.998456 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:01:12.998430 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19947c84_8671_40d9_8124_26148f6491fb.slice/crio-3145194bfa48d40d36c729a5ee66c450cc103e728f528a0431b88542fd5e2e1f WatchSource:0}: Error finding container 3145194bfa48d40d36c729a5ee66c450cc103e728f528a0431b88542fd5e2e1f: Status 404 returned error can't find the container with id 3145194bfa48d40d36c729a5ee66c450cc103e728f528a0431b88542fd5e2e1f Jul 15 14:01:13.140958 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.140882 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-s6fqz"] Jul 15 14:01:13.145588 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.145561 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.148451 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.148423 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 14:01:13.148599 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.148573 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8m8gn\"" Jul 15 14:01:13.153787 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.153768 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-s6fqz"] Jul 15 14:01:13.279449 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.279417 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qflv\" (UniqueName: \"kubernetes.io/projected/501a2552-af60-4325-ac18-7615b83bcadb-kube-api-access-2qflv\") pod \"payload-processing-f9b8d9b9b-s6fqz\" (UID: \"501a2552-af60-4325-ac18-7615b83bcadb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.279610 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.279467 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/501a2552-af60-4325-ac18-7615b83bcadb-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-s6fqz\" (UID: \"501a2552-af60-4325-ac18-7615b83bcadb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.380650 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.380616 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qflv\" (UniqueName: \"kubernetes.io/projected/501a2552-af60-4325-ac18-7615b83bcadb-kube-api-access-2qflv\") pod \"payload-processing-f9b8d9b9b-s6fqz\" (UID: \"501a2552-af60-4325-ac18-7615b83bcadb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.380809 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.380666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/501a2552-af60-4325-ac18-7615b83bcadb-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-s6fqz\" (UID: \"501a2552-af60-4325-ac18-7615b83bcadb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.381345 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.381325 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/501a2552-af60-4325-ac18-7615b83bcadb-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-s6fqz\" (UID: \"501a2552-af60-4325-ac18-7615b83bcadb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.390185 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.390161 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qflv\" (UniqueName: \"kubernetes.io/projected/501a2552-af60-4325-ac18-7615b83bcadb-kube-api-access-2qflv\") pod \"payload-processing-f9b8d9b9b-s6fqz\" (UID: \"501a2552-af60-4325-ac18-7615b83bcadb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.457271 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.457170 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:13.597686 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.597660 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-s6fqz"] Jul 15 14:01:13.599807 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:01:13.599781 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501a2552_af60_4325_ac18_7615b83bcadb.slice/crio-1c720b4c79125c3215acb913ab92a71d7d273794221ee1071b7e0a15a9c22d1b WatchSource:0}: Error finding container 1c720b4c79125c3215acb913ab92a71d7d273794221ee1071b7e0a15a9c22d1b: Status 404 returned error can't find the container with id 1c720b4c79125c3215acb913ab92a71d7d273794221ee1071b7e0a15a9c22d1b Jul 15 14:01:13.971688 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.971314 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" event={"ID":"501a2552-af60-4325-ac18-7615b83bcadb","Type":"ContainerStarted","Data":"1c720b4c79125c3215acb913ab92a71d7d273794221ee1071b7e0a15a9c22d1b"} Jul 15 14:01:13.977761 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:13.977718 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" event={"ID":"19947c84-8671-40d9-8124-26148f6491fb","Type":"ContainerStarted","Data":"3145194bfa48d40d36c729a5ee66c450cc103e728f528a0431b88542fd5e2e1f"} Jul 15 14:01:15.990469 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:15.990390 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" event={"ID":"501a2552-af60-4325-ac18-7615b83bcadb","Type":"ContainerStarted","Data":"eced8b73992712d64755ef18f76d8b080767e3470998f01c4579a0f27dc12dd7"} Jul 15 14:01:15.990943 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:15.990657 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:15.991982 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:15.991956 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" event={"ID":"19947c84-8671-40d9-8124-26148f6491fb","Type":"ContainerStarted","Data":"9ff501400b3fbbb9b97d48fc764a8b135ff9467ebddc6eae313288c721ab37d6"} Jul 15 14:01:15.992099 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:15.992013 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:16.013449 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:16.013410 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" podStartSLOduration=0.762220065 podStartE2EDuration="3.013395136s" totalImagesPullingTime="2.251175071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:01:16.01008269 +0000 UTC m=+614.447691482" watchObservedRunningTime="2026-07-15 14:01:16.013395136 +0000 UTC m=+614.451003928" Jul 15 14:01:21.020652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.020561 2584 generic.go:356] "Generic (PLEG): container finished" podID="c637485c-0a7e-45da-9344-046ab1f44771" containerID="29fe2ea3831408831c3c2246072fc520f91cdb2a206e944188b32bb07b2cbaa2" exitCode=0 Jul 15 14:01:21.020652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.020634 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" event={"ID":"c637485c-0a7e-45da-9344-046ab1f44771","Type":"ContainerDied","Data":"29fe2ea3831408831c3c2246072fc520f91cdb2a206e944188b32bb07b2cbaa2"} Jul 15 14:01:21.021071 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.021001 2584 scope.go:117] "RemoveContainer" containerID="29fe2ea3831408831c3c2246072fc520f91cdb2a206e944188b32bb07b2cbaa2" Jul 15 14:01:21.021999 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.021971 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:01:21.022067 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.022044 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:01:21.022105 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.022077 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:01:21.043640 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:21.043599 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" podStartSLOduration=6.185880455 podStartE2EDuration="9.043584469s" totalImagesPullingTime="2.857704014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:01:16.047075236 +0000 UTC m=+614.484684028" watchObservedRunningTime="2026-07-15 14:01:21.043584469 +0000 UTC m=+619.481193264" Jul 15 14:01:22.014677 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:22.014640 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d75896c4d-r7dbl" Jul 15 14:01:22.026774 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:22.026748 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" event={"ID":"c637485c-0a7e-45da-9344-046ab1f44771","Type":"ContainerStarted","Data":"175cb90f3be04e7a9c7695f5ba2174359563bbae361d26c16fada30389b5c1f5"} Jul 15 14:01:22.221117 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:22.221079 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 14:01:22.222257 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:22.222229 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jgfff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 15 14:01:22.222381 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:22.222281 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" podUID="c637485c-0a7e-45da-9344-046ab1f44771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 15 14:01:23.221515 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:23.221479 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jgfff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 15 14:01:23.221908 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:23.221535 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" podUID="c637485c-0a7e-45da-9344-046ab1f44771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 15 14:01:23.387787 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:23.387751 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 14:01:24.220745 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:24.220669 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jgfff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 15 14:01:24.220745 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:24.220727 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" podUID="c637485c-0a7e-45da-9344-046ab1f44771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 15 14:01:25.221748 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:25.221712 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jgfff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 15 14:01:25.222196 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:25.221768 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" podUID="c637485c-0a7e-45da-9344-046ab1f44771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 15 14:01:26.221363 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:26.221323 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jgfff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 15 14:01:26.221548 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:26.221389 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" podUID="c637485c-0a7e-45da-9344-046ab1f44771" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 15 14:01:27.000052 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:27.000021 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-s6fqz" Jul 15 14:01:27.233078 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:27.233047 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 14:01:27.234205 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:27.234179 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jgfff" Jul 15 14:01:38.099771 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.099737 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc"] Jul 15 14:01:38.104095 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.104074 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.107991 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.107962 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 14:01:38.108141 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.108039 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 14:01:38.108141 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.108099 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-czstb\"" Jul 15 14:01:38.108318 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.108157 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 15 14:01:38.120277 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.120249 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc"] Jul 15 14:01:38.220760 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.220723 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.220927 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.220771 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttfvd\" (UniqueName: \"kubernetes.io/projected/027b2e2b-3c0d-45d5-a956-d2167a262861-kube-api-access-ttfvd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.220927 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.220815 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.220927 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.220912 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027b2e2b-3c0d-45d5-a956-d2167a262861-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.221276 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.220958 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.221276 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.220987 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.321748 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.321710 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027b2e2b-3c0d-45d5-a956-d2167a262861-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.321928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.321772 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.321928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.321807 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.321928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.321839 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.321928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.321879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttfvd\" (UniqueName: \"kubernetes.io/projected/027b2e2b-3c0d-45d5-a956-d2167a262861-kube-api-access-ttfvd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.321928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.321910 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.322407 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.322373 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.322538 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.322383 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.322538 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.322442 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.324143 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.324121 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027b2e2b-3c0d-45d5-a956-d2167a262861-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.324402 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.324387 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027b2e2b-3c0d-45d5-a956-d2167a262861-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.330533 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.330499 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttfvd\" (UniqueName: \"kubernetes.io/projected/027b2e2b-3c0d-45d5-a956-d2167a262861-kube-api-access-ttfvd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc\" (UID: \"027b2e2b-3c0d-45d5-a956-d2167a262861\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.423264 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.423163 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:38.565685 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.565662 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc"] Jul 15 14:01:38.567246 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:01:38.567219 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027b2e2b_3c0d_45d5_a956_d2167a262861.slice/crio-f61c0e02ca3075a581d974348f8f6d8cc15b52b7d01780bc56eb2a091c16bf40 WatchSource:0}: Error finding container f61c0e02ca3075a581d974348f8f6d8cc15b52b7d01780bc56eb2a091c16bf40: Status 404 returned error can't find the container with id f61c0e02ca3075a581d974348f8f6d8cc15b52b7d01780bc56eb2a091c16bf40 Jul 15 14:01:38.568625 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:38.568608 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 14:01:39.133716 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:39.133633 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" event={"ID":"027b2e2b-3c0d-45d5-a956-d2167a262861","Type":"ContainerStarted","Data":"f61c0e02ca3075a581d974348f8f6d8cc15b52b7d01780bc56eb2a091c16bf40"} Jul 15 14:01:50.191026 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:50.190988 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" event={"ID":"027b2e2b-3c0d-45d5-a956-d2167a262861","Type":"ContainerStarted","Data":"1699193d35bc8561d863cd852213e3da6418a6b5a4db365950d7ae924ec07890"} Jul 15 14:01:50.211960 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:50.211887 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" podStartSLOduration=1.520271519 podStartE2EDuration="12.211867357s" totalImagesPullingTime="10.691595838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:01:50.211789293 +0000 UTC m=+648.649398086" watchObservedRunningTime="2026-07-15 14:01:50.211867357 +0000 UTC m=+648.649476151" Jul 15 14:01:58.424173 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:58.424119 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:58.424771 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:58.424207 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:58.436895 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:58.436872 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:01:59.247646 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:01:59.247615 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc" Jul 15 14:02:15.302672 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.302631 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh"] Jul 15 14:02:15.309493 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.309468 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.312664 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.312638 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 14:02:15.319860 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.319338 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh"] Jul 15 14:02:15.444248 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.444215 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.444457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.444324 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.444457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.444369 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02b4d95-a1ea-4183-8b7b-728a39a4de14-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.444457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.444392 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjtv8\" (UniqueName: \"kubernetes.io/projected/d02b4d95-a1ea-4183-8b7b-728a39a4de14-kube-api-access-cjtv8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.444457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.444438 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.444620 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.444533 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545314 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545271 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545509 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545509 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02b4d95-a1ea-4183-8b7b-728a39a4de14-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545509 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545390 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjtv8\" (UniqueName: \"kubernetes.io/projected/d02b4d95-a1ea-4183-8b7b-728a39a4de14-kube-api-access-cjtv8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545509 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545415 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545509 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545462 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545842 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545815 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545911 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545876 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.545911 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.545895 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.547602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.547580 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02b4d95-a1ea-4183-8b7b-728a39a4de14-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.547862 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.547844 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02b4d95-a1ea-4183-8b7b-728a39a4de14-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.555769 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.555713 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjtv8\" (UniqueName: \"kubernetes.io/projected/d02b4d95-a1ea-4183-8b7b-728a39a4de14-kube-api-access-cjtv8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jflvh\" (UID: \"d02b4d95-a1ea-4183-8b7b-728a39a4de14\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.622952 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.622920 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:15.761646 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:15.761616 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh"] Jul 15 14:02:15.763568 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:02:15.763541 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd02b4d95_a1ea_4183_8b7b_728a39a4de14.slice/crio-72834d056d33d58328db0ffec3002b74b3cb77dc426b8e4f3fdb07c84934bec1 WatchSource:0}: Error finding container 72834d056d33d58328db0ffec3002b74b3cb77dc426b8e4f3fdb07c84934bec1: Status 404 returned error can't find the container with id 72834d056d33d58328db0ffec3002b74b3cb77dc426b8e4f3fdb07c84934bec1 Jul 15 14:02:16.316732 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:16.316639 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" event={"ID":"d02b4d95-a1ea-4183-8b7b-728a39a4de14","Type":"ContainerStarted","Data":"78cefe03ecc452ffa304d88a19be811e4cb2a3174082d1f1518859db092b13e3"} Jul 15 14:02:16.316732 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:16.316698 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" event={"ID":"d02b4d95-a1ea-4183-8b7b-728a39a4de14","Type":"ContainerStarted","Data":"72834d056d33d58328db0ffec3002b74b3cb77dc426b8e4f3fdb07c84934bec1"} Jul 15 14:02:16.336900 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:16.336841 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" podStartSLOduration=1.098878518 podStartE2EDuration="1.336822705s" totalImagesPullingTime="237.944187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:02:16.335527404 +0000 UTC m=+674.773136196" watchObservedRunningTime="2026-07-15 14:02:16.336822705 +0000 UTC m=+674.774431497" Jul 15 14:02:25.623200 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:25.623161 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:25.623721 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:25.623317 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:25.636368 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:25.636345 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:02:26.377111 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:02:26.377082 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jflvh" Jul 15 14:03:16.699408 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.699328 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5876bcc5cb-pv6sr"] Jul 15 14:03:16.703275 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.703252 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.706225 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.706200 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q8l6d\"" Jul 15 14:03:16.706225 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.706200 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 14:03:16.707070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.707049 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 14:03:16.707212 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.707185 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 14:03:16.707351 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.707280 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 14:03:16.707351 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.707158 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 14:03:16.707351 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.707130 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 14:03:16.707519 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.707199 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 14:03:16.713714 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.713695 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 14:03:16.714819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.714801 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5876bcc5cb-pv6sr"] Jul 15 14:03:16.817599 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817571 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-oauth-config\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.817599 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817605 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-service-ca\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.817797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817631 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-trusted-ca-bundle\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.817797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817711 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs8sl\" (UniqueName: \"kubernetes.io/projected/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-kube-api-access-fs8sl\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.817797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817737 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-oauth-serving-cert\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.817797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817755 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-config\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.817922 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.817830 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-serving-cert\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919209 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-trusted-ca-bundle\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs8sl\" (UniqueName: \"kubernetes.io/projected/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-kube-api-access-fs8sl\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-oauth-serving-cert\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919427 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-config\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919632 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919478 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-serving-cert\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919632 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919515 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-oauth-config\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.919632 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.919536 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-service-ca\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.920256 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.920228 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-oauth-serving-cert\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.920256 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.920239 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-config\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.920427 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.920266 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-service-ca\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.920470 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.920432 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-trusted-ca-bundle\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.923023 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.922979 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-oauth-config\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.923191 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.923036 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-console-serving-cert\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:16.929997 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:16.929185 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs8sl\" (UniqueName: \"kubernetes.io/projected/e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0-kube-api-access-fs8sl\") pod \"console-5876bcc5cb-pv6sr\" (UID: \"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0\") " pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:17.014617 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:17.014531 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:17.153639 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:17.153616 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5876bcc5cb-pv6sr"] Jul 15 14:03:17.155741 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:03:17.155714 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode149cbc9_ce22_4e6b_a4de_78f1fb9c6ad0.slice/crio-4eddef6c2cdf22a1914ed931b5894ec63c644d40bca64aa00fa54bcaf22a065e WatchSource:0}: Error finding container 4eddef6c2cdf22a1914ed931b5894ec63c644d40bca64aa00fa54bcaf22a065e: Status 404 returned error can't find the container with id 4eddef6c2cdf22a1914ed931b5894ec63c644d40bca64aa00fa54bcaf22a065e Jul 15 14:03:17.610167 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:17.610134 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5876bcc5cb-pv6sr" event={"ID":"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0","Type":"ContainerStarted","Data":"5bc66aa2ef757b6c60ca569c062c223a27fed424528a4e315c71b22d12c2a332"} Jul 15 14:03:17.610383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:17.610182 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5876bcc5cb-pv6sr" event={"ID":"e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0","Type":"ContainerStarted","Data":"4eddef6c2cdf22a1914ed931b5894ec63c644d40bca64aa00fa54bcaf22a065e"} Jul 15 14:03:17.630314 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:17.630253 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5876bcc5cb-pv6sr" podStartSLOduration=1.630237654 podStartE2EDuration="1.630237654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:03:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:03:17.628428107 +0000 UTC m=+736.066036892" watchObservedRunningTime="2026-07-15 14:03:17.630237654 +0000 UTC m=+736.067846446" Jul 15 14:03:27.015510 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:27.015478 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:27.015904 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:27.015548 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:27.020258 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:27.020238 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:27.661912 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:27.661886 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5876bcc5cb-pv6sr" Jul 15 14:03:33.146354 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.146322 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57864c88fb-2rcr8"] Jul 15 14:03:33.149841 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.149825 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.159734 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.159705 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57864c88fb-2rcr8"] Jul 15 14:03:33.162461 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.162435 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5c833a98-9507-4386-b9d5-20041ec59999-tls-cert\") pod \"authorino-57864c88fb-2rcr8\" (UID: \"5c833a98-9507-4386-b9d5-20041ec59999\") " pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.162588 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.162489 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6wt2\" (UniqueName: \"kubernetes.io/projected/5c833a98-9507-4386-b9d5-20041ec59999-kube-api-access-w6wt2\") pod \"authorino-57864c88fb-2rcr8\" (UID: \"5c833a98-9507-4386-b9d5-20041ec59999\") " pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.263959 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.263921 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5c833a98-9507-4386-b9d5-20041ec59999-tls-cert\") pod \"authorino-57864c88fb-2rcr8\" (UID: \"5c833a98-9507-4386-b9d5-20041ec59999\") " pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.264127 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.263968 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6wt2\" (UniqueName: \"kubernetes.io/projected/5c833a98-9507-4386-b9d5-20041ec59999-kube-api-access-w6wt2\") pod \"authorino-57864c88fb-2rcr8\" (UID: \"5c833a98-9507-4386-b9d5-20041ec59999\") " pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.266358 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.266333 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5c833a98-9507-4386-b9d5-20041ec59999-tls-cert\") pod \"authorino-57864c88fb-2rcr8\" (UID: \"5c833a98-9507-4386-b9d5-20041ec59999\") " pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.273952 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.273927 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6wt2\" (UniqueName: \"kubernetes.io/projected/5c833a98-9507-4386-b9d5-20041ec59999-kube-api-access-w6wt2\") pod \"authorino-57864c88fb-2rcr8\" (UID: \"5c833a98-9507-4386-b9d5-20041ec59999\") " pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.460398 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.460313 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57864c88fb-2rcr8" Jul 15 14:03:33.607359 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.607325 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57864c88fb-2rcr8"] Jul 15 14:03:33.610182 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:03:33.610153 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c833a98_9507_4386_b9d5_20041ec59999.slice/crio-4b7bbcf460be65ad1935921602fc262737fef24c1ae5d63094f68ea64ff01002 WatchSource:0}: Error finding container 4b7bbcf460be65ad1935921602fc262737fef24c1ae5d63094f68ea64ff01002: Status 404 returned error can't find the container with id 4b7bbcf460be65ad1935921602fc262737fef24c1ae5d63094f68ea64ff01002 Jul 15 14:03:33.687793 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:33.687748 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57864c88fb-2rcr8" event={"ID":"5c833a98-9507-4386-b9d5-20041ec59999","Type":"ContainerStarted","Data":"4b7bbcf460be65ad1935921602fc262737fef24c1ae5d63094f68ea64ff01002"} Jul 15 14:03:34.697756 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:34.697708 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57864c88fb-2rcr8" event={"ID":"5c833a98-9507-4386-b9d5-20041ec59999","Type":"ContainerStarted","Data":"3184dd153c59404a2532584dc5648be3e67a362abe6ba15976baa0f9224a1c08"} Jul 15 14:03:34.717649 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:34.717595 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57864c88fb-2rcr8" podStartSLOduration=1.099846227 podStartE2EDuration="1.717578464s" totalImagesPullingTime="617.732237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:03:34.715661903 +0000 UTC m=+753.153270702" watchObservedRunningTime="2026-07-15 14:03:34.717578464 +0000 UTC m=+753.155187256" Jul 15 14:03:34.777062 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:34.777026 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594b9d8797-4glwk"] Jul 15 14:03:34.777516 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:34.777480 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-594b9d8797-4glwk" podUID="0b7527c6-80bf-4af4-bc14-9d94e8682217" containerName="authorino" containerID="cri-o://282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d" gracePeriod=30 Jul 15 14:03:35.055716 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.055689 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 14:03:35.085209 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.085155 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkvxq\" (UniqueName: \"kubernetes.io/projected/0b7527c6-80bf-4af4-bc14-9d94e8682217-kube-api-access-jkvxq\") pod \"0b7527c6-80bf-4af4-bc14-9d94e8682217\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " Jul 15 14:03:35.085428 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.085321 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b7527c6-80bf-4af4-bc14-9d94e8682217-tls-cert\") pod \"0b7527c6-80bf-4af4-bc14-9d94e8682217\" (UID: \"0b7527c6-80bf-4af4-bc14-9d94e8682217\") " Jul 15 14:03:35.087974 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.087719 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7527c6-80bf-4af4-bc14-9d94e8682217-kube-api-access-jkvxq" (OuterVolumeSpecName: "kube-api-access-jkvxq") pod "0b7527c6-80bf-4af4-bc14-9d94e8682217" (UID: "0b7527c6-80bf-4af4-bc14-9d94e8682217"). InnerVolumeSpecName "kube-api-access-jkvxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:03:35.100185 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.100149 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b7527c6-80bf-4af4-bc14-9d94e8682217-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0b7527c6-80bf-4af4-bc14-9d94e8682217" (UID: "0b7527c6-80bf-4af4-bc14-9d94e8682217"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 14:03:35.186785 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.186744 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkvxq\" (UniqueName: \"kubernetes.io/projected/0b7527c6-80bf-4af4-bc14-9d94e8682217-kube-api-access-jkvxq\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:03:35.186785 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.186785 2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b7527c6-80bf-4af4-bc14-9d94e8682217-tls-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:03:35.704908 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.704871 2584 generic.go:356] "Generic (PLEG): container finished" podID="0b7527c6-80bf-4af4-bc14-9d94e8682217" containerID="282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d" exitCode=0 Jul 15 14:03:35.705427 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.704924 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594b9d8797-4glwk" Jul 15 14:03:35.705427 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.704957 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594b9d8797-4glwk" event={"ID":"0b7527c6-80bf-4af4-bc14-9d94e8682217","Type":"ContainerDied","Data":"282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d"} Jul 15 14:03:35.705427 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.705000 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594b9d8797-4glwk" event={"ID":"0b7527c6-80bf-4af4-bc14-9d94e8682217","Type":"ContainerDied","Data":"723b969c1bd28249e7c6476b862474a99bfc57135b470edb083625fe9d29b5c2"} Jul 15 14:03:35.705427 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.705019 2584 scope.go:117] "RemoveContainer" containerID="282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d" Jul 15 14:03:35.714340 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.714286 2584 scope.go:117] "RemoveContainer" containerID="282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d" Jul 15 14:03:35.714644 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:03:35.714616 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d\": container with ID starting with 282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d not found: ID does not exist" containerID="282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d" Jul 15 14:03:35.714769 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.714660 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d"} err="failed to get container status \"282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d\": rpc error: code = NotFound desc = could not find container \"282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d\": container with ID starting with 282313d7291bf31deaa661bf21c6442af402ba69d5224492771f77b9d7da3b5d not found: ID does not exist" Jul 15 14:03:35.727463 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.727437 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594b9d8797-4glwk"] Jul 15 14:03:35.732013 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:35.731981 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-594b9d8797-4glwk"] Jul 15 14:03:36.009789 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:03:36.009693 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b7527c6-80bf-4af4-bc14-9d94e8682217" path="/var/lib/kubelet/pods/0b7527c6-80bf-4af4-bc14-9d94e8682217/volumes" Jul 15 14:04:46.072684 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.072578 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-688d755c47-6xlgm"] Jul 15 14:04:46.073267 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.072916 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-688d755c47-6xlgm" podUID="8930f726-be31-4031-a3fc-961049b1d892" containerName="manager" containerID="cri-o://7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6" gracePeriod=30 Jul 15 14:04:46.330209 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.330146 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:04:46.366254 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.366223 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4smcj\" (UniqueName: \"kubernetes.io/projected/8930f726-be31-4031-a3fc-961049b1d892-kube-api-access-4smcj\") pod \"8930f726-be31-4031-a3fc-961049b1d892\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " Jul 15 14:04:46.366465 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.366267 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert\") pod \"8930f726-be31-4031-a3fc-961049b1d892\" (UID: \"8930f726-be31-4031-a3fc-961049b1d892\") " Jul 15 14:04:46.368929 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.368897 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8930f726-be31-4031-a3fc-961049b1d892" (UID: "8930f726-be31-4031-a3fc-961049b1d892"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 14:04:46.369452 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.369423 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8930f726-be31-4031-a3fc-961049b1d892-kube-api-access-4smcj" (OuterVolumeSpecName: "kube-api-access-4smcj") pod "8930f726-be31-4031-a3fc-961049b1d892" (UID: "8930f726-be31-4031-a3fc-961049b1d892"). InnerVolumeSpecName "kube-api-access-4smcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:04:46.467243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.467208 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8930f726-be31-4031-a3fc-961049b1d892-webhook-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:04:46.467243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:46.467241 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4smcj\" (UniqueName: \"kubernetes.io/projected/8930f726-be31-4031-a3fc-961049b1d892-kube-api-access-4smcj\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:04:47.074877 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.074834 2584 generic.go:356] "Generic (PLEG): container finished" podID="8930f726-be31-4031-a3fc-961049b1d892" containerID="7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6" exitCode=0 Jul 15 14:04:47.075320 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.074895 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688d755c47-6xlgm" Jul 15 14:04:47.075320 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.074916 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688d755c47-6xlgm" event={"ID":"8930f726-be31-4031-a3fc-961049b1d892","Type":"ContainerDied","Data":"7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6"} Jul 15 14:04:47.075320 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.075003 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688d755c47-6xlgm" event={"ID":"8930f726-be31-4031-a3fc-961049b1d892","Type":"ContainerDied","Data":"57d36b9a2980b35c0a99733faeaac3f9f16f596c437beeeb8745c42085e5eb9e"} Jul 15 14:04:47.075320 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.075037 2584 scope.go:117] "RemoveContainer" containerID="7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6" Jul 15 14:04:47.084968 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.084951 2584 scope.go:117] "RemoveContainer" containerID="7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6" Jul 15 14:04:47.085222 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:04:47.085204 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6\": container with ID starting with 7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6 not found: ID does not exist" containerID="7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6" Jul 15 14:04:47.085275 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.085231 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6"} err="failed to get container status \"7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6\": rpc error: code = NotFound desc = could not find container \"7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6\": container with ID starting with 7450e5d52c6a23716b06b02679724e8623e16f23a59128311423bd7459bc36d6 not found: ID does not exist" Jul 15 14:04:47.098809 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.098782 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-688d755c47-6xlgm"] Jul 15 14:04:47.102544 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.102519 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-688d755c47-6xlgm"] Jul 15 14:04:47.888445 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888403 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-688d755c47-tm2lw"] Jul 15 14:04:47.888817 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888803 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7527c6-80bf-4af4-bc14-9d94e8682217" containerName="authorino" Jul 15 14:04:47.888817 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888816 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7527c6-80bf-4af4-bc14-9d94e8682217" containerName="authorino" Jul 15 14:04:47.888964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888829 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8930f726-be31-4031-a3fc-961049b1d892" containerName="manager" Jul 15 14:04:47.888964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888839 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8930f726-be31-4031-a3fc-961049b1d892" containerName="manager" Jul 15 14:04:47.888964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888913 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8930f726-be31-4031-a3fc-961049b1d892" containerName="manager" Jul 15 14:04:47.888964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.888922 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7527c6-80bf-4af4-bc14-9d94e8682217" containerName="authorino" Jul 15 14:04:47.893462 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.893439 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:47.897542 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.897519 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 14:04:47.897941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.897753 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m6j56\"" Jul 15 14:04:47.897941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.897785 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 14:04:47.900725 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.900704 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-688d755c47-tm2lw"] Jul 15 14:04:47.979181 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.979148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22a2b338-2612-49d6-8c1b-3d73d18d6696-webhook-cert\") pod \"maas-controller-688d755c47-tm2lw\" (UID: \"22a2b338-2612-49d6-8c1b-3d73d18d6696\") " pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:47.979181 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:47.979179 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24fg7\" (UniqueName: \"kubernetes.io/projected/22a2b338-2612-49d6-8c1b-3d73d18d6696-kube-api-access-24fg7\") pod \"maas-controller-688d755c47-tm2lw\" (UID: \"22a2b338-2612-49d6-8c1b-3d73d18d6696\") " pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:48.008698 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.008674 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8930f726-be31-4031-a3fc-961049b1d892" path="/var/lib/kubelet/pods/8930f726-be31-4031-a3fc-961049b1d892/volumes" Jul 15 14:04:48.080203 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.080177 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22a2b338-2612-49d6-8c1b-3d73d18d6696-webhook-cert\") pod \"maas-controller-688d755c47-tm2lw\" (UID: \"22a2b338-2612-49d6-8c1b-3d73d18d6696\") " pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:48.080619 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.080213 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24fg7\" (UniqueName: \"kubernetes.io/projected/22a2b338-2612-49d6-8c1b-3d73d18d6696-kube-api-access-24fg7\") pod \"maas-controller-688d755c47-tm2lw\" (UID: \"22a2b338-2612-49d6-8c1b-3d73d18d6696\") " pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:48.082680 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.082650 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22a2b338-2612-49d6-8c1b-3d73d18d6696-webhook-cert\") pod \"maas-controller-688d755c47-tm2lw\" (UID: \"22a2b338-2612-49d6-8c1b-3d73d18d6696\") " pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:48.089129 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.089105 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24fg7\" (UniqueName: \"kubernetes.io/projected/22a2b338-2612-49d6-8c1b-3d73d18d6696-kube-api-access-24fg7\") pod \"maas-controller-688d755c47-tm2lw\" (UID: \"22a2b338-2612-49d6-8c1b-3d73d18d6696\") " pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:48.205928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.205869 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:48.340898 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:48.340874 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-688d755c47-tm2lw"] Jul 15 14:04:48.343316 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:04:48.343277 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a2b338_2612_49d6_8c1b_3d73d18d6696.slice/crio-e30c33bdcb86d1c9abe61f9f86e278dc1387eb9cc3ff00a4116ef80d92e0321f WatchSource:0}: Error finding container e30c33bdcb86d1c9abe61f9f86e278dc1387eb9cc3ff00a4116ef80d92e0321f: Status 404 returned error can't find the container with id e30c33bdcb86d1c9abe61f9f86e278dc1387eb9cc3ff00a4116ef80d92e0321f Jul 15 14:04:49.087913 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:49.087878 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688d755c47-tm2lw" event={"ID":"22a2b338-2612-49d6-8c1b-3d73d18d6696","Type":"ContainerStarted","Data":"74627c505cb64d7759b7382b634809aff8e2e11819a36d6bdf41a5346b820d10"} Jul 15 14:04:49.088327 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:49.087941 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688d755c47-tm2lw" event={"ID":"22a2b338-2612-49d6-8c1b-3d73d18d6696","Type":"ContainerStarted","Data":"e30c33bdcb86d1c9abe61f9f86e278dc1387eb9cc3ff00a4116ef80d92e0321f"} Jul 15 14:04:49.088327 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:49.087974 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:04:49.106871 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:04:49.106811 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-688d755c47-tm2lw" podStartSLOduration=1.744684508 podStartE2EDuration="2.106794131s" totalImagesPullingTime="362.109623ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:04:49.104839142 +0000 UTC m=+827.542447935" watchObservedRunningTime="2026-07-15 14:04:49.106794131 +0000 UTC m=+827.544402939" Jul 15 14:05:00.094968 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:05:00.094937 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-688d755c47-tm2lw" Jul 15 14:06:01.976774 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:06:01.976749 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:06:01.977317 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:06:01.977193 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:06:01.985276 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:06:01.985258 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:06:01.985400 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:06:01.985364 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:09:32.638937 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:32.638900 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb"] Jul 15 14:09:32.639553 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:32.639140 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" podUID="e5ebfa81-2287-4a58-bf52-efe8660aff90" containerName="manager" containerID="cri-o://fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1" gracePeriod=10 Jul 15 14:09:34.699457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.699424 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 14:09:34.760587 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.760553 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ebfa81-2287-4a58-bf52-efe8660aff90-extensions-socket-volume\") pod \"e5ebfa81-2287-4a58-bf52-efe8660aff90\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " Jul 15 14:09:34.760761 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.760716 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwplh\" (UniqueName: \"kubernetes.io/projected/e5ebfa81-2287-4a58-bf52-efe8660aff90-kube-api-access-lwplh\") pod \"e5ebfa81-2287-4a58-bf52-efe8660aff90\" (UID: \"e5ebfa81-2287-4a58-bf52-efe8660aff90\") " Jul 15 14:09:34.760937 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.760916 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ebfa81-2287-4a58-bf52-efe8660aff90-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e5ebfa81-2287-4a58-bf52-efe8660aff90" (UID: "e5ebfa81-2287-4a58-bf52-efe8660aff90"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:09:34.762769 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.762737 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ebfa81-2287-4a58-bf52-efe8660aff90-kube-api-access-lwplh" (OuterVolumeSpecName: "kube-api-access-lwplh") pod "e5ebfa81-2287-4a58-bf52-efe8660aff90" (UID: "e5ebfa81-2287-4a58-bf52-efe8660aff90"). InnerVolumeSpecName "kube-api-access-lwplh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:09:34.861949 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.861905 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ebfa81-2287-4a58-bf52-efe8660aff90-extensions-socket-volume\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:09:34.861949 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:34.861939 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwplh\" (UniqueName: \"kubernetes.io/projected/e5ebfa81-2287-4a58-bf52-efe8660aff90-kube-api-access-lwplh\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:09:35.530253 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.530216 2584 generic.go:356] "Generic (PLEG): container finished" podID="e5ebfa81-2287-4a58-bf52-efe8660aff90" containerID="fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1" exitCode=0 Jul 15 14:09:35.530487 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.530262 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" Jul 15 14:09:35.530487 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.530312 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" event={"ID":"e5ebfa81-2287-4a58-bf52-efe8660aff90","Type":"ContainerDied","Data":"fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1"} Jul 15 14:09:35.530487 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.530374 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb" event={"ID":"e5ebfa81-2287-4a58-bf52-efe8660aff90","Type":"ContainerDied","Data":"1751d655f363b22f2667ee985d1b99492abf2210e3b841ba56840e05f66161fe"} Jul 15 14:09:35.530487 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.530396 2584 scope.go:117] "RemoveContainer" containerID="fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1" Jul 15 14:09:35.539850 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.539831 2584 scope.go:117] "RemoveContainer" containerID="fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1" Jul 15 14:09:35.540133 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:09:35.540112 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1\": container with ID starting with fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1 not found: ID does not exist" containerID="fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1" Jul 15 14:09:35.540183 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.540141 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1"} err="failed to get container status \"fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1\": rpc error: code = NotFound desc = could not find container \"fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1\": container with ID starting with fef909e2e526fe52a8426e1483931e415b9cd8293cff240fde21319db3c18de1 not found: ID does not exist" Jul 15 14:09:35.553735 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.553704 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb"] Jul 15 14:09:35.558008 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:35.557984 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpmb"] Jul 15 14:09:36.008180 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:09:36.008138 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ebfa81-2287-4a58-bf52-efe8660aff90" path="/var/lib/kubelet/pods/e5ebfa81-2287-4a58-bf52-efe8660aff90/volumes" Jul 15 14:10:33.850741 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.850643 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj"] Jul 15 14:10:33.851355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.851271 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ebfa81-2287-4a58-bf52-efe8660aff90" containerName="manager" Jul 15 14:10:33.851355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.851313 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ebfa81-2287-4a58-bf52-efe8660aff90" containerName="manager" Jul 15 14:10:33.851486 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.851449 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ebfa81-2287-4a58-bf52-efe8660aff90" containerName="manager" Jul 15 14:10:33.854539 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.854518 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:33.857951 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.857933 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nwrzz\"" Jul 15 14:10:33.866636 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.866614 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj"] Jul 15 14:10:33.926864 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.926825 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10016cb2-44b9-460d-9281-bd4fe2b596b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zk9hj\" (UID: \"10016cb2-44b9-460d-9281-bd4fe2b596b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:33.927031 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:33.926927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5kc4\" (UniqueName: \"kubernetes.io/projected/10016cb2-44b9-460d-9281-bd4fe2b596b8-kube-api-access-q5kc4\") pod \"kuadrant-operator-controller-manager-86878c769f-zk9hj\" (UID: \"10016cb2-44b9-460d-9281-bd4fe2b596b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.028218 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.028184 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5kc4\" (UniqueName: \"kubernetes.io/projected/10016cb2-44b9-460d-9281-bd4fe2b596b8-kube-api-access-q5kc4\") pod \"kuadrant-operator-controller-manager-86878c769f-zk9hj\" (UID: \"10016cb2-44b9-460d-9281-bd4fe2b596b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.028423 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.028330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10016cb2-44b9-460d-9281-bd4fe2b596b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zk9hj\" (UID: \"10016cb2-44b9-460d-9281-bd4fe2b596b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.028727 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.028705 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10016cb2-44b9-460d-9281-bd4fe2b596b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zk9hj\" (UID: \"10016cb2-44b9-460d-9281-bd4fe2b596b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.041086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.041057 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5kc4\" (UniqueName: \"kubernetes.io/projected/10016cb2-44b9-460d-9281-bd4fe2b596b8-kube-api-access-q5kc4\") pod \"kuadrant-operator-controller-manager-86878c769f-zk9hj\" (UID: \"10016cb2-44b9-460d-9281-bd4fe2b596b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.165267 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.165181 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.311377 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.311355 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj"] Jul 15 14:10:34.313442 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:10:34.313415 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10016cb2_44b9_460d_9281_bd4fe2b596b8.slice/crio-8752844fa0b4bece413ac5090dbf1c3b183c9c6080e00929fa2e38477071d6f1 WatchSource:0}: Error finding container 8752844fa0b4bece413ac5090dbf1c3b183c9c6080e00929fa2e38477071d6f1: Status 404 returned error can't find the container with id 8752844fa0b4bece413ac5090dbf1c3b183c9c6080e00929fa2e38477071d6f1 Jul 15 14:10:34.822519 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.822484 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" event={"ID":"10016cb2-44b9-460d-9281-bd4fe2b596b8","Type":"ContainerStarted","Data":"871fe84a29a2f5e456cc7a08aceb261b238a22245464318586748dc1eff577dc"} Jul 15 14:10:34.822721 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.822534 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" event={"ID":"10016cb2-44b9-460d-9281-bd4fe2b596b8","Type":"ContainerStarted","Data":"8752844fa0b4bece413ac5090dbf1c3b183c9c6080e00929fa2e38477071d6f1"} Jul 15 14:10:34.822721 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.822609 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:10:34.844290 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:34.844241 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" podStartSLOduration=1.844219983 podStartE2EDuration="1.844219983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:10:34.843029442 +0000 UTC m=+1173.280638236" watchObservedRunningTime="2026-07-15 14:10:34.844219983 +0000 UTC m=+1173.281828779" Jul 15 14:10:45.830532 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:10:45.830499 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zk9hj" Jul 15 14:11:02.007543 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:11:02.007508 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:11:02.009922 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:11:02.009898 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:11:02.015683 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:11:02.015660 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:11:02.018079 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:11:02.018061 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:15:00.164276 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.164192 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z"] Jul 15 14:15:00.167765 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.167748 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" Jul 15 14:15:00.171097 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.171071 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w42np\"" Jul 15 14:15:00.177762 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.177738 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" Jul 15 14:15:00.194177 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.194148 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z"] Jul 15 14:15:00.341849 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.341817 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z"] Jul 15 14:15:00.343371 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:15:00.343340 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e315b9_9a9f_42b5_9d34_3747e3f56076.slice/crio-2b59edacae8f5c243890f38e0f368271bbfdc0b530543f89a6ac89fdb8f0fb06 WatchSource:0}: Error finding container 2b59edacae8f5c243890f38e0f368271bbfdc0b530543f89a6ac89fdb8f0fb06: Status 404 returned error can't find the container with id 2b59edacae8f5c243890f38e0f368271bbfdc0b530543f89a6ac89fdb8f0fb06 Jul 15 14:15:00.345340 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:00.345321 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 14:15:01.202855 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:01.202819 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" event={"ID":"69e315b9-9a9f-42b5-9d34-3747e3f56076","Type":"ContainerStarted","Data":"2b59edacae8f5c243890f38e0f368271bbfdc0b530543f89a6ac89fdb8f0fb06"} Jul 15 14:15:03.216624 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:03.216589 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" event={"ID":"69e315b9-9a9f-42b5-9d34-3747e3f56076","Type":"ContainerStarted","Data":"fb2790f1cb0226b4ca3ef329aa18ac5a54cacde7bb8000529f75e68089efef39"} Jul 15 14:15:04.223681 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:04.223641 2584 generic.go:356] "Generic (PLEG): container finished" podID="69e315b9-9a9f-42b5-9d34-3747e3f56076" containerID="fb2790f1cb0226b4ca3ef329aa18ac5a54cacde7bb8000529f75e68089efef39" exitCode=0 Jul 15 14:15:04.224165 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:04.223730 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" event={"ID":"69e315b9-9a9f-42b5-9d34-3747e3f56076","Type":"ContainerDied","Data":"fb2790f1cb0226b4ca3ef329aa18ac5a54cacde7bb8000529f75e68089efef39"} Jul 15 14:15:05.387227 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:05.387199 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" Jul 15 14:15:06.235783 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:06.235753 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" Jul 15 14:15:06.235783 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:06.235768 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-d7f7z" event={"ID":"69e315b9-9a9f-42b5-9d34-3747e3f56076","Type":"ContainerDied","Data":"2b59edacae8f5c243890f38e0f368271bbfdc0b530543f89a6ac89fdb8f0fb06"} Jul 15 14:15:06.236012 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:15:06.235822 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b59edacae8f5c243890f38e0f368271bbfdc0b530543f89a6ac89fdb8f0fb06" Jul 15 14:16:02.043796 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:02.043770 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:16:02.047675 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:02.047640 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:16:02.052757 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:02.052737 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:16:02.055940 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:02.055922 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:16:59.187245 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.187209 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb"] Jul 15 14:16:59.187853 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.187832 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69e315b9-9a9f-42b5-9d34-3747e3f56076" containerName="cleanup" Jul 15 14:16:59.187897 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.187858 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="69e315b9-9a9f-42b5-9d34-3747e3f56076" containerName="cleanup" Jul 15 14:16:59.188015 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.188003 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="69e315b9-9a9f-42b5-9d34-3747e3f56076" containerName="cleanup" Jul 15 14:16:59.191689 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.191670 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.195319 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.195283 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-af5d844d-openshift-default-dockercfg-x8n89\"" Jul 15 14:16:59.225124 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1830e39a-481d-453e-8a8b-6b8052626d97-istio-podinfo\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.225312 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225138 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1830e39a-481d-453e-8a8b-6b8052626d97-istiod-ca-cert\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.225312 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225200 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-data\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.225312 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225245 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-socket\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.225312 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225273 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-istio-token\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.226166 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225368 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvvt\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-kube-api-access-srvvt\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.226166 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225416 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-credential-socket\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.226166 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225451 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-envoy\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.226166 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.225514 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-certs\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.229484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.229459 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb"] Jul 15 14:16:59.326135 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326100 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-istio-token\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326148 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvvt\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-kube-api-access-srvvt\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326167 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-credential-socket\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326184 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-envoy\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-certs\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326239 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1830e39a-481d-453e-8a8b-6b8052626d97-istio-podinfo\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326260 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1830e39a-481d-453e-8a8b-6b8052626d97-istiod-ca-cert\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326288 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-data\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326361 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-socket\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326818 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326634 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-credential-socket\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326818 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326658 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-certs\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326818 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326736 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-socket\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.326991 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.326823 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-data\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.327132 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.327109 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1830e39a-481d-453e-8a8b-6b8052626d97-istiod-ca-cert\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.328653 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.328627 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-envoy\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.328852 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.328833 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1830e39a-481d-453e-8a8b-6b8052626d97-istio-podinfo\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.338903 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.338879 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-istio-token\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.341819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.341799 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvvt\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-kube-api-access-srvvt\") pod \"e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.502578 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.502496 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:16:59.655440 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.655413 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb"] Jul 15 14:16:59.657400 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:16:59.657370 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1830e39a_481d_453e_8a8b_6b8052626d97.slice/crio-d2bfaf2f5bc896fef91b772252142f9b090e9fd77ddeec146325805b44b75a2d WatchSource:0}: Error finding container d2bfaf2f5bc896fef91b772252142f9b090e9fd77ddeec146325805b44b75a2d: Status 404 returned error can't find the container with id d2bfaf2f5bc896fef91b772252142f9b090e9fd77ddeec146325805b44b75a2d Jul 15 14:16:59.659410 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.659380 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:16:59.659488 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.659441 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:16:59.659488 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.659468 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:16:59.846643 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.846608 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" event={"ID":"1830e39a-481d-453e-8a8b-6b8052626d97","Type":"ContainerStarted","Data":"afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e"} Jul 15 14:16:59.846834 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.846667 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" event={"ID":"1830e39a-481d-453e-8a8b-6b8052626d97","Type":"ContainerStarted","Data":"d2bfaf2f5bc896fef91b772252142f9b090e9fd77ddeec146325805b44b75a2d"} Jul 15 14:16:59.892080 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:16:59.892029 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podStartSLOduration=0.892013174 podStartE2EDuration="892.013174ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:16:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:16:59.890351797 +0000 UTC m=+1558.327960589" watchObservedRunningTime="2026-07-15 14:16:59.892013174 +0000 UTC m=+1558.329621965" Jul 15 14:17:00.503154 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:00.503117 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:00.504641 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:00.504614 2584 patch_prober.go:28] interesting pod/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 15 14:17:00.504756 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:00.504664 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 15 14:17:01.507759 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:01.507720 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:01.856724 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:01.856694 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:01.857745 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:01.857727 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:12.915022 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:12.914986 2584 generic.go:356] "Generic (PLEG): container finished" podID="1830e39a-481d-453e-8a8b-6b8052626d97" containerID="afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e" exitCode=0 Jul 15 14:17:12.915477 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:12.915058 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" event={"ID":"1830e39a-481d-453e-8a8b-6b8052626d97","Type":"ContainerDied","Data":"afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e"} Jul 15 14:17:12.915522 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:12.915481 2584 scope.go:117] "RemoveContainer" containerID="afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e" Jul 15 14:17:12.916462 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:12.916435 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:12.916531 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:12.916503 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:12.916570 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:12.916536 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:13.922186 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:13.922150 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" event={"ID":"1830e39a-481d-453e-8a8b-6b8052626d97","Type":"ContainerStarted","Data":"4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000"} Jul 15 14:17:14.502884 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:14.502842 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:14.504054 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:14.504026 2584 patch_prober.go:28] interesting pod/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 15 14:17:14.504172 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:14.504076 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 15 14:17:15.503794 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:15.503754 2584 patch_prober.go:28] interesting pod/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 15 14:17:15.504172 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:15.503817 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 15 14:17:16.503333 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:16.503275 2584 patch_prober.go:28] interesting pod/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 15 14:17:16.503517 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:16.503355 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 15 14:17:16.857290 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:16.857247 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:17.503199 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:17.503159 2584 patch_prober.go:28] interesting pod/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 15 14:17:17.503415 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:17.503258 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 15 14:17:18.506966 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:18.506937 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:18.508145 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:18.508128 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:30.782433 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:30.782393 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb"] Jul 15 14:17:30.782936 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:30.782589 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" containerID="cri-o://4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000" gracePeriod=30 Jul 15 14:17:31.492143 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.492111 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7"] Jul 15 14:17:31.495969 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.495951 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.501797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.501771 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-30d2f55b-gw-openshift-default-dockercfg-sthtk\"" Jul 15 14:17:31.540258 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540203 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istiod-ca-cert\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540258 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540241 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-envoy\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540258 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-podinfo\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540564 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540284 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-data\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540564 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540475 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-token\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540564 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540544 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-credential-socket\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540701 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540651 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-certs\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540701 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540685 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skd5l\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-kube-api-access-skd5l\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.540798 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.540725 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-socket\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.561241 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.561213 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7"] Jul 15 14:17:31.641946 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.641907 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-token\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642159 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.641963 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-credential-socket\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642159 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.641994 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-certs\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642280 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642164 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skd5l\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-kube-api-access-skd5l\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642280 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-socket\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642445 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642336 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istiod-ca-cert\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642445 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-envoy\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642445 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642401 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-podinfo\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642445 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642433 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-data\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642613 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642402 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-certs\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642613 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642494 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-credential-socket\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642680 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642622 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-socket\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.642870 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.642832 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-data\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.643188 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.643165 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istiod-ca-cert\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.645189 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.645158 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-envoy\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.645332 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.645273 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-podinfo\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.660234 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.660155 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skd5l\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-kube-api-access-skd5l\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.668410 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.668374 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-token\") pod \"e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.806191 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.806093 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:31.985817 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.985772 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7"] Jul 15 14:17:31.989001 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:17:31.988963 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e1cb20_7993_4da9_9c25_420aa4af2f2f.slice/crio-7a13cc1a5e43ebad9967a0a3d1d78b35cdf395986c7b8bd3a24940e2b8bb9f0d WatchSource:0}: Error finding container 7a13cc1a5e43ebad9967a0a3d1d78b35cdf395986c7b8bd3a24940e2b8bb9f0d: Status 404 returned error can't find the container with id 7a13cc1a5e43ebad9967a0a3d1d78b35cdf395986c7b8bd3a24940e2b8bb9f0d Jul 15 14:17:31.991375 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.991344 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:31.991457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.991411 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:31.991457 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:31.991440 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:32.021426 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:32.021390 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" event={"ID":"a6e1cb20-7993-4da9-9c25-420aa4af2f2f","Type":"ContainerStarted","Data":"7a13cc1a5e43ebad9967a0a3d1d78b35cdf395986c7b8bd3a24940e2b8bb9f0d"} Jul 15 14:17:33.028954 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:33.028913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" event={"ID":"a6e1cb20-7993-4da9-9c25-420aa4af2f2f","Type":"ContainerStarted","Data":"feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186"} Jul 15 14:17:33.059730 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:33.059683 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podStartSLOduration=2.059668024 podStartE2EDuration="2.059668024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:17:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:17:33.058383351 +0000 UTC m=+1591.495992155" watchObservedRunningTime="2026-07-15 14:17:33.059668024 +0000 UTC m=+1591.497276815" Jul 15 14:17:33.510011 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:33.509971 2584 patch_prober.go:28] interesting pod/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 14:17:33.510220 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:33.510047 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 14:17:33.807236 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:33.807127 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:33.812454 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:33.812426 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:34.036202 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:34.036176 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:34.037356 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:34.037338 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:36.041145 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.041071 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:36.049005 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.048979 2584 generic.go:356] "Generic (PLEG): container finished" podID="1830e39a-481d-453e-8a8b-6b8052626d97" containerID="4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000" exitCode=0 Jul 15 14:17:36.049139 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.049042 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" Jul 15 14:17:36.049139 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.049058 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" event={"ID":"1830e39a-481d-453e-8a8b-6b8052626d97","Type":"ContainerDied","Data":"4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000"} Jul 15 14:17:36.049139 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.049102 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb" event={"ID":"1830e39a-481d-453e-8a8b-6b8052626d97","Type":"ContainerDied","Data":"d2bfaf2f5bc896fef91b772252142f9b090e9fd77ddeec146325805b44b75a2d"} Jul 15 14:17:36.049139 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.049127 2584 scope.go:117] "RemoveContainer" containerID="4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000" Jul 15 14:17:36.058113 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.058094 2584 scope.go:117] "RemoveContainer" containerID="afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e" Jul 15 14:17:36.067892 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.067856 2584 scope.go:117] "RemoveContainer" containerID="4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000" Jul 15 14:17:36.068235 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:17:36.068208 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000\": container with ID starting with 4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000 not found: ID does not exist" containerID="4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000" Jul 15 14:17:36.068335 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.068247 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000"} err="failed to get container status \"4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000\": rpc error: code = NotFound desc = could not find container \"4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000\": container with ID starting with 4629fb28434a7f81bdcb6b3617354d7e2ce5ffa94be07189befe0db6e64c6000 not found: ID does not exist" Jul 15 14:17:36.068335 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.068270 2584 scope.go:117] "RemoveContainer" containerID="afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e" Jul 15 14:17:36.068617 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:17:36.068592 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e\": container with ID starting with afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e not found: ID does not exist" containerID="afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e" Jul 15 14:17:36.068685 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.068627 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e"} err="failed to get container status \"afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e\": rpc error: code = NotFound desc = could not find container \"afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e\": container with ID starting with afdb43e39edea99aab79aeb53b25dde084723d8dca122e825e25876f6590041e not found: ID does not exist" Jul 15 14:17:36.091322 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091273 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-istio-token\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091527 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091351 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-certs\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091527 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091380 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1830e39a-481d-453e-8a8b-6b8052626d97-istiod-ca-cert\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091527 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091419 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-envoy\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091527 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091477 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-socket\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091527 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091512 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-credential-socket\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091815 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091577 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1830e39a-481d-453e-8a8b-6b8052626d97-istio-podinfo\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091815 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091622 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-data\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.091815 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.091656 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srvvt\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-kube-api-access-srvvt\") pod \"1830e39a-481d-453e-8a8b-6b8052626d97\" (UID: \"1830e39a-481d-453e-8a8b-6b8052626d97\") " Jul 15 14:17:36.092378 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.092203 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:17:36.092730 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.092563 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:17:36.093098 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.092855 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:17:36.093098 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.092941 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1830e39a-481d-453e-8a8b-6b8052626d97-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:17:36.094327 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.093859 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-istio-token" (OuterVolumeSpecName: "istio-token") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:17:36.094327 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.093880 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-data" (OuterVolumeSpecName: "istio-data") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:17:36.094644 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.094614 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:17:36.095040 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.095020 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-kube-api-access-srvvt" (OuterVolumeSpecName: "kube-api-access-srvvt") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "kube-api-access-srvvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:17:36.096069 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.096045 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1830e39a-481d-453e-8a8b-6b8052626d97-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1830e39a-481d-453e-8a8b-6b8052626d97" (UID: "1830e39a-481d-453e-8a8b-6b8052626d97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:17:36.192819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192783 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.192819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192811 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.192819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192820 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1830e39a-481d-453e-8a8b-6b8052626d97-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.192819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192829 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.193086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192837 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.193086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192844 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.193086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192852 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1830e39a-481d-453e-8a8b-6b8052626d97-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.193086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192860 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1830e39a-481d-453e-8a8b-6b8052626d97-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.193086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.192868 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srvvt\" (UniqueName: \"kubernetes.io/projected/1830e39a-481d-453e-8a8b-6b8052626d97-kube-api-access-srvvt\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:17:36.378965 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.378932 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb"] Jul 15 14:17:36.383993 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:36.383966 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-af5d844d-openshift-default-546bdbbbdc-r6ljb"] Jul 15 14:17:38.008745 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:38.008707 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" path="/var/lib/kubelet/pods/1830e39a-481d-453e-8a8b-6b8052626d97/volumes" Jul 15 14:17:47.108130 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:47.108098 2584 generic.go:356] "Generic (PLEG): container finished" podID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerID="feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186" exitCode=0 Jul 15 14:17:47.108570 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:47.108165 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" event={"ID":"a6e1cb20-7993-4da9-9c25-420aa4af2f2f","Type":"ContainerDied","Data":"feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186"} Jul 15 14:17:47.108570 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:47.108502 2584 scope.go:117] "RemoveContainer" containerID="feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186" Jul 15 14:17:47.109534 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:47.109505 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:47.109600 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:47.109584 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:47.109641 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:47.109627 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:17:48.115081 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:48.115048 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" event={"ID":"a6e1cb20-7993-4da9-9c25-420aa4af2f2f","Type":"ContainerStarted","Data":"c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2"} Jul 15 14:17:48.806310 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:48.806263 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:48.807354 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:48.807325 2584 patch_prober.go:28] interesting pod/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 15 14:17:48.807487 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:48.807378 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 15 14:17:49.036933 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:49.036887 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:49.807257 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:49.807215 2584 patch_prober.go:28] interesting pod/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 15 14:17:49.807698 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:49.807289 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 15 14:17:50.806684 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:50.806644 2584 patch_prober.go:28] interesting pod/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 15 14:17:50.806880 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:50.806720 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 15 14:17:51.806765 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:51.806729 2584 patch_prober.go:28] interesting pod/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 15 14:17:51.807162 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:51.806784 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 15 14:17:52.810472 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:52.810441 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:17:52.811520 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:17:52.811496 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:18:37.445057 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:37.445023 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7"] Jul 15 14:18:37.445580 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:37.445246 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" containerID="cri-o://c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2" gracePeriod=30 Jul 15 14:18:37.813877 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:37.813798 2584 patch_prober.go:28] interesting pod/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 14:18:37.813877 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:37.813851 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 14:18:42.703084 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.703058 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:18:42.832022 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.831993 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istiod-ca-cert\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832044 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-token\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832085 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-socket\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832105 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-podinfo\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832143 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-certs\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832169 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-credential-socket\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832202 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-data\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832289 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-envoy\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832348 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skd5l\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-kube-api-access-skd5l\") pod \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\" (UID: \"a6e1cb20-7993-4da9-9c25-420aa4af2f2f\") " Jul 15 14:18:42.832484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832461 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:18:42.832484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832471 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:18:42.832742 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832654 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:18:42.832742 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832701 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.832742 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832719 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.832915 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.832827 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:18:42.833681 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.833654 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:18:42.834602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.834561 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:18:42.834602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.834587 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:18:42.834874 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.834853 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-kube-api-access-skd5l" (OuterVolumeSpecName: "kube-api-access-skd5l") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "kube-api-access-skd5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:18:42.835098 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.835078 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a6e1cb20-7993-4da9-9c25-420aa4af2f2f" (UID: "a6e1cb20-7993-4da9-9c25-420aa4af2f2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:18:42.933875 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933847 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.933875 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933870 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skd5l\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-kube-api-access-skd5l\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.933875 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933879 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.934107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933887 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.934107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933895 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.934107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933903 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:42.934107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:42.933911 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1cb20-7993-4da9-9c25-420aa4af2f2f-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:18:43.395185 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.395155 2584 generic.go:356] "Generic (PLEG): container finished" podID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerID="c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2" exitCode=0 Jul 15 14:18:43.395378 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.395191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" event={"ID":"a6e1cb20-7993-4da9-9c25-420aa4af2f2f","Type":"ContainerDied","Data":"c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2"} Jul 15 14:18:43.395378 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.395200 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" Jul 15 14:18:43.395378 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.395235 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7" event={"ID":"a6e1cb20-7993-4da9-9c25-420aa4af2f2f","Type":"ContainerDied","Data":"7a13cc1a5e43ebad9967a0a3d1d78b35cdf395986c7b8bd3a24940e2b8bb9f0d"} Jul 15 14:18:43.395378 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.395256 2584 scope.go:117] "RemoveContainer" containerID="c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2" Jul 15 14:18:43.404704 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.404686 2584 scope.go:117] "RemoveContainer" containerID="feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186" Jul 15 14:18:43.412736 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.412720 2584 scope.go:117] "RemoveContainer" containerID="c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2" Jul 15 14:18:43.412953 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:18:43.412935 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2\": container with ID starting with c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2 not found: ID does not exist" containerID="c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2" Jul 15 14:18:43.413001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.412960 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2"} err="failed to get container status \"c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2\": rpc error: code = NotFound desc = could not find container \"c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2\": container with ID starting with c231b12666c98ad03528380be76490633ba6d32102719dfa69ab40e4aa09a3e2 not found: ID does not exist" Jul 15 14:18:43.413001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.412977 2584 scope.go:117] "RemoveContainer" containerID="feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186" Jul 15 14:18:43.413177 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:18:43.413164 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186\": container with ID starting with feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186 not found: ID does not exist" containerID="feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186" Jul 15 14:18:43.413251 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.413179 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186"} err="failed to get container status \"feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186\": rpc error: code = NotFound desc = could not find container \"feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186\": container with ID starting with feea7870962034619a6eb898755eaf607e0652689d0f2fd368ffd0c2e2f6f186 not found: ID does not exist" Jul 15 14:18:43.418999 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.418980 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7"] Jul 15 14:18:43.427489 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:43.427465 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-30d2f55b-gw-openshift-default-56d7997b4c-jjlk7"] Jul 15 14:18:44.007592 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.007560 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" path="/var/lib/kubelet/pods/a6e1cb20-7993-4da9-9c25-420aa4af2f2f/volumes" Jul 15 14:18:44.602681 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.602652 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn"] Jul 15 14:18:44.603159 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603139 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" Jul 15 14:18:44.603274 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603162 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" Jul 15 14:18:44.603274 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603186 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" Jul 15 14:18:44.603274 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603194 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" Jul 15 14:18:44.603458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603335 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" Jul 15 14:18:44.603458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603353 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" Jul 15 14:18:44.603458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.603361 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" Jul 15 14:18:44.608042 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.608020 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.610924 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.610906 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dd81e643-openshift-default-dockercfg-tlrdv\"" Jul 15 14:18:44.624130 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.624107 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn"] Jul 15 14:18:44.753024 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.752982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5j56\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-kube-api-access-d5j56\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753196 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753033 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-socket\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753196 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753136 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-podinfo\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-token\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753258 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istiod-ca-cert\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753404 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753344 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-envoy\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753404 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753361 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-certs\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753404 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753391 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-data\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.753513 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.753409 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-credential-socket\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.854746 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.854658 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-envoy\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.854746 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.854703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-certs\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.854964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.854846 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-data\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.854964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.854889 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-credential-socket\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.854964 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.854927 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5j56\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-kube-api-access-d5j56\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855128 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.854967 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-socket\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855128 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855025 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-certs\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855128 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855031 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-podinfo\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-token\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855169 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-data\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855191 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istiod-ca-cert\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855252 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-credential-socket\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855495 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855371 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-socket\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.855807 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.855789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istiod-ca-cert\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.856959 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.856942 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-envoy\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.857335 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.857288 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-podinfo\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.867768 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.867743 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-token\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.868953 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.868933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5j56\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-kube-api-access-d5j56\") pod \"e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:44.918782 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:44.918756 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:45.061774 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.061752 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn"] Jul 15 14:18:45.064127 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:18:45.064104 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c3f6bf_4f78_40ff_a85a_54c196d898ad.slice/crio-f3325b649e88c38c31580bceb72dc5a7f19f899924dcd2edc607524e28901450 WatchSource:0}: Error finding container f3325b649e88c38c31580bceb72dc5a7f19f899924dcd2edc607524e28901450: Status 404 returned error can't find the container with id f3325b649e88c38c31580bceb72dc5a7f19f899924dcd2edc607524e28901450 Jul 15 14:18:45.066064 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.066032 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:18:45.066152 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.066134 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:18:45.066214 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.066180 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:18:45.408422 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.408326 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" event={"ID":"e0c3f6bf-4f78-40ff-a85a-54c196d898ad","Type":"ContainerStarted","Data":"9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b"} Jul 15 14:18:45.408422 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.408389 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" event={"ID":"e0c3f6bf-4f78-40ff-a85a-54c196d898ad","Type":"ContainerStarted","Data":"f3325b649e88c38c31580bceb72dc5a7f19f899924dcd2edc607524e28901450"} Jul 15 14:18:45.439166 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.439119 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" podStartSLOduration=1.439104283 podStartE2EDuration="1.439104283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:18:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:18:45.436921194 +0000 UTC m=+1663.874529981" watchObservedRunningTime="2026-07-15 14:18:45.439104283 +0000 UTC m=+1663.876713075" Jul 15 14:18:45.919255 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:45.919222 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:46.923726 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:46.923697 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:47.419982 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:47.419952 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:47.420822 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:47.420804 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:56.476204 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:56.476168 2584 generic.go:356] "Generic (PLEG): container finished" podID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerID="9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b" exitCode=0 Jul 15 14:18:56.476637 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:56.476242 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" event={"ID":"e0c3f6bf-4f78-40ff-a85a-54c196d898ad","Type":"ContainerDied","Data":"9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b"} Jul 15 14:18:56.476690 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:56.476663 2584 scope.go:117] "RemoveContainer" containerID="9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b" Jul 15 14:18:56.477750 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:56.477723 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:18:56.477821 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:56.477808 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:18:56.477860 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:56.477844 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:18:57.482736 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:57.482696 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" event={"ID":"e0c3f6bf-4f78-40ff-a85a-54c196d898ad","Type":"ContainerStarted","Data":"54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb"} Jul 15 14:18:57.919593 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:57.919558 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:18:57.920928 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:57.920905 2584 patch_prober.go:28] interesting pod/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 15 14:18:57.921030 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:57.920951 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 15 14:18:58.919316 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:58.919256 2584 patch_prober.go:28] interesting pod/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 15 14:18:58.919718 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:58.919335 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 15 14:18:59.919452 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:59.919418 2584 patch_prober.go:28] interesting pod/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 15 14:18:59.919830 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:18:59.919476 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 15 14:19:00.919445 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:19:00.919412 2584 patch_prober.go:28] interesting pod/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 15 14:19:00.919634 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:19:00.919477 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 15 14:19:01.922987 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:19:01.922956 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:19:01.923436 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:19:01.923235 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:19:01.923881 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:19:01.923865 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:20:07.287855 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.287818 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn"] Jul 15 14:20:07.288415 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.288017 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" containerID="cri-o://54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb" gracePeriod=30 Jul 15 14:20:07.771119 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.771090 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj"] Jul 15 14:20:07.771578 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.771564 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" Jul 15 14:20:07.771627 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.771581 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1830e39a-481d-453e-8a8b-6b8052626d97" containerName="istio-proxy" Jul 15 14:20:07.771627 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.771607 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" Jul 15 14:20:07.771627 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.771613 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" Jul 15 14:20:07.771722 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.771677 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e1cb20-7993-4da9-9c25-420aa4af2f2f" containerName="istio-proxy" Jul 15 14:20:07.774635 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.774619 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.778164 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.778144 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-55a6f882-openshift-default-dockercfg-6dzxg\"" Jul 15 14:20:07.821776 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.821751 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj"] Jul 15 14:20:07.920026 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.919992 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-certs\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920179 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920049 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-podinfo\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920179 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920084 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-credential-socket\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920258 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920181 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-envoy\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920258 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-data\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920346 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920269 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-token\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920341 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-socket\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920418 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920382 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf4jk\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-kube-api-access-pf4jk\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:07.920418 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:07.920406 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istiod-ca-cert\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.021775 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021706 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-socket\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.021775 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021745 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf4jk\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-kube-api-access-pf4jk\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.021775 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021769 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istiod-ca-cert\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021795 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-certs\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021820 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-podinfo\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021839 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-credential-socket\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.021984 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-envoy\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022027 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-data\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022334 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022153 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-socket\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022334 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022151 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-token\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022334 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022210 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-certs\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022511 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022428 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-data\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022574 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022550 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-credential-socket\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.022717 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.022698 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istiod-ca-cert\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.024145 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.024120 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-envoy\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.024332 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.024317 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-podinfo\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.031510 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.031489 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-token\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.031972 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.031952 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf4jk\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-kube-api-access-pf4jk\") pod \"e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.084859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.084834 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:08.225907 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.225881 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj"] Jul 15 14:20:08.228214 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:20:08.228187 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a79ed19_cd8c_4ef4_8c25_0a5065d01ad1.slice/crio-d6a59b0d57622803533ec13219d3bd61483dd6ed3053862a98528662e3d39879 WatchSource:0}: Error finding container d6a59b0d57622803533ec13219d3bd61483dd6ed3053862a98528662e3d39879: Status 404 returned error can't find the container with id d6a59b0d57622803533ec13219d3bd61483dd6ed3053862a98528662e3d39879 Jul 15 14:20:08.230479 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.230437 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:20:08.230573 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.230530 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:20:08.230635 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.230574 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:20:08.855073 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.855038 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" event={"ID":"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1","Type":"ContainerStarted","Data":"5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5"} Jul 15 14:20:08.855480 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.855099 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" event={"ID":"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1","Type":"ContainerStarted","Data":"d6a59b0d57622803533ec13219d3bd61483dd6ed3053862a98528662e3d39879"} Jul 15 14:20:08.887087 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:08.887039 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podStartSLOduration=1.887007527 podStartE2EDuration="1.887007527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:20:08.883996041 +0000 UTC m=+1747.321604843" watchObservedRunningTime="2026-07-15 14:20:08.887007527 +0000 UTC m=+1747.324616318" Jul 15 14:20:09.085238 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:09.085203 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:09.089853 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:09.089828 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:09.861957 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:09.861925 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:09.863001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:09.862977 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:12.555479 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.555406 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:20:12.665975 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.665949 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-socket\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666138 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666031 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-token\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666138 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666094 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-envoy\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666348 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666141 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-podinfo\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666348 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666169 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5j56\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-kube-api-access-d5j56\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666348 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666199 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istiod-ca-cert\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666348 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666236 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-data\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666348 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666274 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-certs\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666694 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666326 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-credential-socket\") pod \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\" (UID: \"e0c3f6bf-4f78-40ff-a85a-54c196d898ad\") " Jul 15 14:20:12.666992 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666588 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:20:12.666992 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666606 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:20:12.666992 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666844 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:20:12.666992 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.666967 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:20:12.667228 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.667133 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.667228 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.667155 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.667228 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.667168 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.667228 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.667182 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.667228 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.667205 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:20:12.668625 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.668588 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-kube-api-access-d5j56" (OuterVolumeSpecName: "kube-api-access-d5j56") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "kube-api-access-d5j56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:20:12.668722 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.668691 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:20:12.668817 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.668799 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:20:12.668905 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.668888 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0c3f6bf-4f78-40ff-a85a-54c196d898ad" (UID: "e0c3f6bf-4f78-40ff-a85a-54c196d898ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:20:12.768319 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.768282 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.768319 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.768319 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.768484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.768329 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5j56\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-kube-api-access-d5j56\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.768484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.768337 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.768484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.768346 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0c3f6bf-4f78-40ff-a85a-54c196d898ad-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:20:12.879776 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.879745 2584 generic.go:356] "Generic (PLEG): container finished" podID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerID="54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb" exitCode=0 Jul 15 14:20:12.879915 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.879797 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" Jul 15 14:20:12.879915 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.879837 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" event={"ID":"e0c3f6bf-4f78-40ff-a85a-54c196d898ad","Type":"ContainerDied","Data":"54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb"} Jul 15 14:20:12.879915 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.879900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn" event={"ID":"e0c3f6bf-4f78-40ff-a85a-54c196d898ad","Type":"ContainerDied","Data":"f3325b649e88c38c31580bceb72dc5a7f19f899924dcd2edc607524e28901450"} Jul 15 14:20:12.880090 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.879930 2584 scope.go:117] "RemoveContainer" containerID="54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb" Jul 15 14:20:12.889853 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.889837 2584 scope.go:117] "RemoveContainer" containerID="9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b" Jul 15 14:20:12.897717 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.897689 2584 scope.go:117] "RemoveContainer" containerID="54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb" Jul 15 14:20:12.897967 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:20:12.897950 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb\": container with ID starting with 54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb not found: ID does not exist" containerID="54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb" Jul 15 14:20:12.898024 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.897976 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb"} err="failed to get container status \"54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb\": rpc error: code = NotFound desc = could not find container \"54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb\": container with ID starting with 54b2646f1f59dc7221ee8d2b24f56851547def4ee688e5d32fc5e4ed413362fb not found: ID does not exist" Jul 15 14:20:12.898024 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.897992 2584 scope.go:117] "RemoveContainer" containerID="9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b" Jul 15 14:20:12.898251 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:20:12.898231 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b\": container with ID starting with 9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b not found: ID does not exist" containerID="9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b" Jul 15 14:20:12.898307 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.898257 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b"} err="failed to get container status \"9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b\": rpc error: code = NotFound desc = could not find container \"9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b\": container with ID starting with 9ad7159fd8ed708e4fde7c4be2f0f67d78e0818e4ff38cd8fe9f79a5964d002b not found: ID does not exist" Jul 15 14:20:12.919768 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.919740 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn"] Jul 15 14:20:12.928513 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:12.928493 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dd81e643-openshift-default-78cb895bb9-w4lkn"] Jul 15 14:20:14.007897 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:14.007863 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" path="/var/lib/kubelet/pods/e0c3f6bf-4f78-40ff-a85a-54c196d898ad/volumes" Jul 15 14:20:20.923421 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:20.923389 2584 generic.go:356] "Generic (PLEG): container finished" podID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerID="5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5" exitCode=0 Jul 15 14:20:20.923863 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:20.923431 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" event={"ID":"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1","Type":"ContainerDied","Data":"5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5"} Jul 15 14:20:20.923863 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:20.923812 2584 scope.go:117] "RemoveContainer" containerID="5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5" Jul 15 14:20:20.924826 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:20.924796 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:20:20.924918 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:20.924862 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:20:20.924918 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:20.924892 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:20:21.930282 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:21.930250 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" event={"ID":"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1","Type":"ContainerStarted","Data":"6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834"} Jul 15 14:20:22.085263 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:22.085227 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:22.086279 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:22.086259 2584 patch_prober.go:28] interesting pod/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 15 14:20:22.086355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:22.086315 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 15 14:20:23.085781 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:23.085749 2584 patch_prober.go:28] interesting pod/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 15 14:20:23.086151 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:23.085798 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 15 14:20:24.085677 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:24.085587 2584 patch_prober.go:28] interesting pod/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 15 14:20:24.085677 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:24.085645 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 15 14:20:24.862709 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:24.862676 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:25.086187 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:25.086156 2584 patch_prober.go:28] interesting pod/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 15 14:20:25.086588 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:25.086207 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 15 14:20:26.089766 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:26.089733 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:20:26.090605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:20:26.090587 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:21:02.077089 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:02.077060 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:21:02.080516 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:02.080492 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:21:02.085237 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:02.085215 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:21:02.088101 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:02.088083 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:21:38.592967 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:38.592915 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj"] Jul 15 14:21:38.593564 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:38.593201 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" containerID="cri-o://6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834" gracePeriod=30 Jul 15 14:21:41.092087 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:41.092048 2584 patch_prober.go:28] interesting pod/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 14:21:41.092482 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:41.092120 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 14:21:43.854030 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.853968 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:21:43.915398 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915360 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-token\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915588 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915474 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-podinfo\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915639 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-socket\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915724 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915702 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-certs\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915779 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915745 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-data\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915849 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915792 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf4jk\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-kube-api-access-pf4jk\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915849 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915833 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-envoy\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915951 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915867 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istiod-ca-cert\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.915951 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915911 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-credential-socket\") pod \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\" (UID: \"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1\") " Jul 15 14:21:43.916059 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.915955 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:21:43.916288 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.916265 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:43.916524 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.916343 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:21:43.916702 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.916675 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:21:43.916826 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.916747 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:21:43.917093 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.917071 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:21:43.918362 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.918284 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:21:43.918465 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.918367 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:21:43.918705 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.918678 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-kube-api-access-pf4jk" (OuterVolumeSpecName: "kube-api-access-pf4jk") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "kube-api-access-pf4jk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:21:43.918831 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:43.918729 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" (UID: "6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:21:44.017840 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017813 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf4jk\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-kube-api-access-pf4jk\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.017840 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017838 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.018061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017847 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.018061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017855 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.018061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017864 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.018061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017875 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.018061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017888 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.018061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.017897 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:21:44.365142 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.365107 2584 generic.go:356] "Generic (PLEG): container finished" podID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerID="6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834" exitCode=0 Jul 15 14:21:44.365341 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.365166 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" event={"ID":"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1","Type":"ContainerDied","Data":"6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834"} Jul 15 14:21:44.365341 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.365179 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" Jul 15 14:21:44.365341 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.365211 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj" event={"ID":"6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1","Type":"ContainerDied","Data":"d6a59b0d57622803533ec13219d3bd61483dd6ed3053862a98528662e3d39879"} Jul 15 14:21:44.365341 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.365234 2584 scope.go:117] "RemoveContainer" containerID="6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834" Jul 15 14:21:44.374248 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.374218 2584 scope.go:117] "RemoveContainer" containerID="5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5" Jul 15 14:21:44.382741 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.382711 2584 scope.go:117] "RemoveContainer" containerID="6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834" Jul 15 14:21:44.383039 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:21:44.383016 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834\": container with ID starting with 6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834 not found: ID does not exist" containerID="6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834" Jul 15 14:21:44.383110 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.383047 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834"} err="failed to get container status \"6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834\": rpc error: code = NotFound desc = could not find container \"6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834\": container with ID starting with 6d9329476ba8022b6cde45e49d444530dd2cfa0cbd3ca94df95897f254369834 not found: ID does not exist" Jul 15 14:21:44.383110 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.383065 2584 scope.go:117] "RemoveContainer" containerID="5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5" Jul 15 14:21:44.383336 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:21:44.383318 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5\": container with ID starting with 5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5 not found: ID does not exist" containerID="5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5" Jul 15 14:21:44.383387 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.383341 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5"} err="failed to get container status \"5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5\": rpc error: code = NotFound desc = could not find container \"5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5\": container with ID starting with 5710de491b3a7e8aa769f265318e1435e37eb300c3839620edc0a329a2edf8c5 not found: ID does not exist" Jul 15 14:21:44.390794 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.390772 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj"] Jul 15 14:21:44.396319 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:44.396279 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-55a6f882-openshift-default-b4767987f-5wwsj"] Jul 15 14:21:46.010272 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:21:46.010240 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" path="/var/lib/kubelet/pods/6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1/volumes" Jul 15 14:23:17.114135 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114103 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6"] Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114503 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114514 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114527 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114533 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114544 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114549 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114616 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114624 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a79ed19-cd8c-4ef4-8c25-0a5065d01ad1" containerName="istio-proxy" Jul 15 14:23:17.114647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.114632 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" Jul 15 14:23:17.117665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.117647 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.120667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.120648 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-625773fa-openshift-default-dockercfg-fnj7k\"" Jul 15 14:23:17.145531 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.145498 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6"] Jul 15 14:23:17.191679 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191642 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dd152b-511d-4f19-9f3c-f69c60f04077-istiod-ca-cert\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.191883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191690 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-envoy\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.191883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191724 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-token\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.191883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191770 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-data\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.191883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191813 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9wtl\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-kube-api-access-j9wtl\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.191883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191835 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-podinfo\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.191883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191854 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-certs\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.192222 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191901 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-socket\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.192222 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.191945 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-credential-socket\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293201 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9wtl\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-kube-api-access-j9wtl\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293249 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-podinfo\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293530 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293279 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-certs\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293530 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293322 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-socket\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293530 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-credential-socket\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293530 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dd152b-511d-4f19-9f3c-f69c60f04077-istiod-ca-cert\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293530 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293508 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-envoy\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293788 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293577 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-token\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293788 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-data\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293788 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293750 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-certs\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293956 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293821 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-socket\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.293956 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.293849 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-credential-socket\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.294068 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.294039 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-data\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.294291 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.294267 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dd152b-511d-4f19-9f3c-f69c60f04077-istiod-ca-cert\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.295784 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.295763 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-envoy\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.295972 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.295957 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-podinfo\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.301996 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.301976 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-token\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.302150 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.302129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9wtl\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-kube-api-access-j9wtl\") pod \"e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.428545 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.428464 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:17.576865 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.576842 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6"] Jul 15 14:23:17.579059 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:23:17.579021 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dd152b_511d_4f19_9f3c_f69c60f04077.slice/crio-a4cfb2bdec2034b294adc44ff0dadb2c486910e3a2cceff0718db63c52a199cb WatchSource:0}: Error finding container a4cfb2bdec2034b294adc44ff0dadb2c486910e3a2cceff0718db63c52a199cb: Status 404 returned error can't find the container with id a4cfb2bdec2034b294adc44ff0dadb2c486910e3a2cceff0718db63c52a199cb Jul 15 14:23:17.582174 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.582127 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:23:17.582350 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.582226 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:23:17.582350 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.582272 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:23:17.851235 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.851201 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" event={"ID":"e8dd152b-511d-4f19-9f3c-f69c60f04077","Type":"ContainerStarted","Data":"a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab"} Jul 15 14:23:17.851422 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.851260 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" event={"ID":"e8dd152b-511d-4f19-9f3c-f69c60f04077","Type":"ContainerStarted","Data":"a4cfb2bdec2034b294adc44ff0dadb2c486910e3a2cceff0718db63c52a199cb"} Jul 15 14:23:17.882676 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:17.882617 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podStartSLOduration=0.882596434 podStartE2EDuration="882.596434ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:23:17.881754296 +0000 UTC m=+1936.319363088" watchObservedRunningTime="2026-07-15 14:23:17.882596434 +0000 UTC m=+1936.320205228" Jul 15 14:23:18.429383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:18.429348 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:18.434883 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:18.434856 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:18.857197 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:18.857162 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:18.858087 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:18.858064 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:33.991233 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:33.989519 2584 patch_prober.go:28] interesting pod/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:41866->10.133.0.62:15021: read: connection reset by peer" start-of-body= Jul 15 14:23:33.991233 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:33.989589 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:41866->10.133.0.62:15021: read: connection reset by peer" Jul 15 14:23:34.945652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:34.945621 2584 generic.go:356] "Generic (PLEG): container finished" podID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerID="a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab" exitCode=0 Jul 15 14:23:34.945820 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:34.945668 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" event={"ID":"e8dd152b-511d-4f19-9f3c-f69c60f04077","Type":"ContainerDied","Data":"a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab"} Jul 15 14:23:34.946029 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:34.946011 2584 scope.go:117] "RemoveContainer" containerID="a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab" Jul 15 14:23:34.947002 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:34.946972 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:23:34.947100 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:34.947038 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:23:34.947100 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:34.947070 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:23:35.956903 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:35.956856 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" event={"ID":"e8dd152b-511d-4f19-9f3c-f69c60f04077","Type":"ContainerStarted","Data":"3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8"} Jul 15 14:23:36.429131 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:36.429102 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:36.429344 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:36.429267 2584 patch_prober.go:28] interesting pod/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 15 14:23:36.429344 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:36.429332 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 15 14:23:37.429777 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:37.429737 2584 patch_prober.go:28] interesting pod/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 15 14:23:37.430152 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:37.429813 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 15 14:23:38.429399 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:38.429365 2584 patch_prober.go:28] interesting pod/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 15 14:23:38.429607 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:38.429424 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 15 14:23:39.428986 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:39.428948 2584 patch_prober.go:28] interesting pod/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 15 14:23:39.429403 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:39.429015 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 15 14:23:40.432664 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:40.432635 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:40.433147 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:40.433002 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:40.433765 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:40.433744 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:23:58.689371 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:58.689274 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6"] Jul 15 14:23:58.689951 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:23:58.689502 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" containerID="cri-o://3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8" gracePeriod=30 Jul 15 14:24:03.943307 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.943275 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:24:03.947283 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947263 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-podinfo\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947408 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947317 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dd152b-511d-4f19-9f3c-f69c60f04077-istiod-ca-cert\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947408 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947357 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-token\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947521 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947428 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-credential-socket\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947521 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947504 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9wtl\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-kube-api-access-j9wtl\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947626 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947543 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-socket\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947730 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947706 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-certs\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947798 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947765 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:24:03.947844 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947775 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:24:03.947844 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947800 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8dd152b-511d-4f19-9f3c-f69c60f04077-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:24:03.947952 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947910 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-data\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.947952 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947929 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:24:03.947952 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.947940 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-envoy\") pod \"e8dd152b-511d-4f19-9f3c-f69c60f04077\" (UID: \"e8dd152b-511d-4f19-9f3c-f69c60f04077\") " Jul 15 14:24:03.948289 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.948267 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8dd152b-511d-4f19-9f3c-f69c60f04077-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:03.948468 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.948446 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:03.948468 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.948468 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:03.948649 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.948481 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:03.948649 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.948627 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:24:03.949617 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.949595 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:24:03.949711 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.949689 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:24:03.949980 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.949963 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-kube-api-access-j9wtl" (OuterVolumeSpecName: "kube-api-access-j9wtl") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "kube-api-access-j9wtl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:24:03.950082 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:03.950063 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8dd152b-511d-4f19-9f3c-f69c60f04077" (UID: "e8dd152b-511d-4f19-9f3c-f69c60f04077"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:24:04.049048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.049015 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:04.049048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.049044 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:04.049218 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.049054 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9wtl\" (UniqueName: \"kubernetes.io/projected/e8dd152b-511d-4f19-9f3c-f69c60f04077-kube-api-access-j9wtl\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:04.049218 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.049063 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:04.049218 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.049073 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8dd152b-511d-4f19-9f3c-f69c60f04077-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:24:04.105873 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.105841 2584 generic.go:356] "Generic (PLEG): container finished" podID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerID="3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8" exitCode=0 Jul 15 14:24:04.106009 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.105880 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" Jul 15 14:24:04.106009 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.105914 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" event={"ID":"e8dd152b-511d-4f19-9f3c-f69c60f04077","Type":"ContainerDied","Data":"3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8"} Jul 15 14:24:04.106009 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.105970 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6" event={"ID":"e8dd152b-511d-4f19-9f3c-f69c60f04077","Type":"ContainerDied","Data":"a4cfb2bdec2034b294adc44ff0dadb2c486910e3a2cceff0718db63c52a199cb"} Jul 15 14:24:04.106009 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.105999 2584 scope.go:117] "RemoveContainer" containerID="3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8" Jul 15 14:24:04.114858 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.114840 2584 scope.go:117] "RemoveContainer" containerID="a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab" Jul 15 14:24:04.122537 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.122523 2584 scope.go:117] "RemoveContainer" containerID="3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8" Jul 15 14:24:04.122801 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:24:04.122785 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8\": container with ID starting with 3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8 not found: ID does not exist" containerID="3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8" Jul 15 14:24:04.122844 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.122818 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8"} err="failed to get container status \"3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8\": rpc error: code = NotFound desc = could not find container \"3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8\": container with ID starting with 3797a9f2a9433fe49b70041cd549343a40cf36dfbb1ef62cddb64b20dd5f55b8 not found: ID does not exist" Jul 15 14:24:04.122844 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.122835 2584 scope.go:117] "RemoveContainer" containerID="a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab" Jul 15 14:24:04.123076 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:24:04.123060 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab\": container with ID starting with a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab not found: ID does not exist" containerID="a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab" Jul 15 14:24:04.123115 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.123083 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab"} err="failed to get container status \"a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab\": rpc error: code = NotFound desc = could not find container \"a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab\": container with ID starting with a8efb170ed8e09101ff8ecf432ca9caa91208c93eb446e138790cfa5962f4dab not found: ID does not exist" Jul 15 14:24:04.131999 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.131977 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6"] Jul 15 14:24:04.137529 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:04.137503 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-625773fa-openshift-default-75b4697b47-h6cr6"] Jul 15 14:24:06.008659 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:06.008628 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" path="/var/lib/kubelet/pods/e8dd152b-511d-4f19-9f3c-f69c60f04077/volumes" Jul 15 14:24:19.090148 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090115 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z"] Jul 15 14:24:19.090602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090587 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" Jul 15 14:24:19.090648 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090604 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" Jul 15 14:24:19.090648 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090619 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" Jul 15 14:24:19.090648 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090625 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" Jul 15 14:24:19.090648 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090637 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" Jul 15 14:24:19.090648 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090643 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" Jul 15 14:24:19.090859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090724 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" Jul 15 14:24:19.090859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.090735 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c3f6bf-4f78-40ff-a85a-54c196d898ad" containerName="istio-proxy" Jul 15 14:24:19.093920 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.093900 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.096872 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.096851 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9e73fa92-openshift-default-dockercfg-pzs9t\"" Jul 15 14:24:19.109836 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.109808 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z"] Jul 15 14:24:19.180806 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180776 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m42nn\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-kube-api-access-m42nn\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.180989 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180842 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-token\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.180989 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180872 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istiod-ca-cert\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.180989 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180897 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-data\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.180989 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180921 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-podinfo\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.180989 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180946 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-socket\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.180989 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.180977 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-certs\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.181273 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.181043 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-envoy\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.181273 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.181105 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-credential-socket\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.281884 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.281845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m42nn\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-kube-api-access-m42nn\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.281910 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-token\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.281937 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istiod-ca-cert\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.281960 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-data\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.281985 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-podinfo\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282011 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-socket\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282045 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-certs\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282419 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282082 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-envoy\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282419 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282148 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-credential-socket\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282526 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-data\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282526 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282494 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-certs\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282642 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282534 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-socket\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282849 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282818 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-credential-socket\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.282984 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.282889 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istiod-ca-cert\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.284380 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.284359 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-envoy\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.284696 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.284674 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-podinfo\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.294232 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.294210 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m42nn\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-kube-api-access-m42nn\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.295709 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.295688 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-token\") pod \"e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.405347 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.405225 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:19.581058 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.580924 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z"] Jul 15 14:24:19.583856 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:24:19.583826 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fded7cb_62f6_4771_aa00_80d27ecf77b6.slice/crio-a31c1af86e37286cad320f9d1687db4a4195ae0db37a0b0f12f0c531f1066c00 WatchSource:0}: Error finding container a31c1af86e37286cad320f9d1687db4a4195ae0db37a0b0f12f0c531f1066c00: Status 404 returned error can't find the container with id a31c1af86e37286cad320f9d1687db4a4195ae0db37a0b0f12f0c531f1066c00 Jul 15 14:24:19.586237 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.586208 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:24:19.586315 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.586277 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:24:19.586360 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:19.586330 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:24:20.189746 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:20.189680 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" event={"ID":"6fded7cb-62f6-4771-aa00-80d27ecf77b6","Type":"ContainerStarted","Data":"308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3"} Jul 15 14:24:20.189746 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:20.189747 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" event={"ID":"6fded7cb-62f6-4771-aa00-80d27ecf77b6","Type":"ContainerStarted","Data":"a31c1af86e37286cad320f9d1687db4a4195ae0db37a0b0f12f0c531f1066c00"} Jul 15 14:24:20.213833 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:20.213777 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podStartSLOduration=1.213760379 podStartE2EDuration="1.213760379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:24:20.212135092 +0000 UTC m=+1998.649743905" watchObservedRunningTime="2026-07-15 14:24:20.213760379 +0000 UTC m=+1998.651369171" Jul 15 14:24:20.405924 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:20.405878 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:20.407548 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:20.407518 2584 patch_prober.go:28] interesting pod/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 15 14:24:20.407696 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:20.407586 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 15 14:24:21.410374 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:21.410342 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:22.201111 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:22.201080 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:22.202092 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:22.202069 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:36.289907 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:36.289871 2584 generic.go:356] "Generic (PLEG): container finished" podID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerID="308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3" exitCode=0 Jul 15 14:24:36.290444 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:36.289947 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" event={"ID":"6fded7cb-62f6-4771-aa00-80d27ecf77b6","Type":"ContainerDied","Data":"308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3"} Jul 15 14:24:36.290444 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:36.290391 2584 scope.go:117] "RemoveContainer" containerID="308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3" Jul 15 14:24:36.291411 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:36.291385 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:24:36.291471 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:36.291450 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:24:36.291510 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:36.291485 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:24:37.201413 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:37.201376 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:37.297176 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:37.297136 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" event={"ID":"6fded7cb-62f6-4771-aa00-80d27ecf77b6","Type":"ContainerStarted","Data":"c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0"} Jul 15 14:24:37.405890 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:37.405861 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:37.406947 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:37.406922 2584 patch_prober.go:28] interesting pod/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 15 14:24:37.407075 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:37.406979 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 15 14:24:38.406085 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:38.406047 2584 patch_prober.go:28] interesting pod/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 15 14:24:38.406515 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:38.406113 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 15 14:24:39.406557 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:39.406522 2584 patch_prober.go:28] interesting pod/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 15 14:24:39.406942 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:39.406583 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 15 14:24:40.406669 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:40.406630 2584 patch_prober.go:28] interesting pod/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 15 14:24:40.407090 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:40.406690 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 15 14:24:41.409612 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:41.409585 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:41.410047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:41.409864 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:24:41.410602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:24:41.410583 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:26:02.107802 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:02.107771 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:26:02.111986 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:02.111965 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:26:02.116628 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:02.116609 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:26:02.119978 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:02.119961 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:26:18.170062 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:18.170025 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z"] Jul 15 14:26:18.170583 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:18.170320 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" containerID="cri-o://c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0" gracePeriod=30 Jul 15 14:26:23.461503 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.461472 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:26:23.588243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588212 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istiod-ca-cert\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588246 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-token\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588498 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588275 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-data\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588498 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588324 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-certs\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588498 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588360 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m42nn\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-kube-api-access-m42nn\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588498 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588407 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-socket\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588498 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588478 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-podinfo\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588765 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588567 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-envoy\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588765 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588593 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-credential-socket\") pod \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\" (UID: \"6fded7cb-62f6-4771-aa00-80d27ecf77b6\") " Jul 15 14:26:23.588876 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588798 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:26:23.588941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588918 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.589001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588936 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:26:23.589001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.588962 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:26:23.589113 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.589096 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:26:23.589327 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.589257 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:26:23.590979 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.590952 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:26:23.591174 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.591145 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:26:23.591402 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.591366 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-kube-api-access-m42nn" (OuterVolumeSpecName: "kube-api-access-m42nn") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "kube-api-access-m42nn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:26:23.591547 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.591522 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6fded7cb-62f6-4771-aa00-80d27ecf77b6" (UID: "6fded7cb-62f6-4771-aa00-80d27ecf77b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690258 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690289 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690313 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690323 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690331 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690339 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fded7cb-62f6-4771-aa00-80d27ecf77b6-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690346 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m42nn\" (UniqueName: \"kubernetes.io/projected/6fded7cb-62f6-4771-aa00-80d27ecf77b6-kube-api-access-m42nn\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.690355 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.690354 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fded7cb-62f6-4771-aa00-80d27ecf77b6-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:26:23.856350 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.856285 2584 generic.go:356] "Generic (PLEG): container finished" podID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerID="c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0" exitCode=0 Jul 15 14:26:23.856519 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.856371 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" Jul 15 14:26:23.856519 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.856379 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" event={"ID":"6fded7cb-62f6-4771-aa00-80d27ecf77b6","Type":"ContainerDied","Data":"c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0"} Jul 15 14:26:23.856519 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.856427 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z" event={"ID":"6fded7cb-62f6-4771-aa00-80d27ecf77b6","Type":"ContainerDied","Data":"a31c1af86e37286cad320f9d1687db4a4195ae0db37a0b0f12f0c531f1066c00"} Jul 15 14:26:23.856519 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.856452 2584 scope.go:117] "RemoveContainer" containerID="c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0" Jul 15 14:26:23.868448 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.868263 2584 scope.go:117] "RemoveContainer" containerID="308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3" Jul 15 14:26:23.878280 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.878256 2584 scope.go:117] "RemoveContainer" containerID="c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0" Jul 15 14:26:23.878699 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:26:23.878679 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0\": container with ID starting with c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0 not found: ID does not exist" containerID="c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0" Jul 15 14:26:23.878804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.878707 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0"} err="failed to get container status \"c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0\": rpc error: code = NotFound desc = could not find container \"c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0\": container with ID starting with c4d2f7ae26c08d9c464a46ba9017e4d0b288b6dc4a9010da097a0eaf45fc09b0 not found: ID does not exist" Jul 15 14:26:23.878804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.878737 2584 scope.go:117] "RemoveContainer" containerID="308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3" Jul 15 14:26:23.879042 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:26:23.879025 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3\": container with ID starting with 308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3 not found: ID does not exist" containerID="308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3" Jul 15 14:26:23.879096 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.879053 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3"} err="failed to get container status \"308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3\": rpc error: code = NotFound desc = could not find container \"308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3\": container with ID starting with 308c7760e668e5fea2368c504bc29a14533157d678844db384869813926096c3 not found: ID does not exist" Jul 15 14:26:23.886236 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.886208 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z"] Jul 15 14:26:23.890839 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:23.890813 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9e73fa92-openshift-default-7ddc59b6ff-5wp6z"] Jul 15 14:26:24.010027 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:26:24.009956 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" path="/var/lib/kubelet/pods/6fded7cb-62f6-4771-aa00-80d27ecf77b6/volumes" Jul 15 14:27:14.836732 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.836694 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s"] Jul 15 14:27:14.837309 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837279 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" Jul 15 14:27:14.837368 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837341 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" Jul 15 14:27:14.837404 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837389 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" Jul 15 14:27:14.837404 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837398 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" Jul 15 14:27:14.837517 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837503 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8dd152b-511d-4f19-9f3c-f69c60f04077" containerName="istio-proxy" Jul 15 14:27:14.837553 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837523 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" Jul 15 14:27:14.837553 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.837535 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fded7cb-62f6-4771-aa00-80d27ecf77b6" containerName="istio-proxy" Jul 15 14:27:14.843604 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.843396 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.850105 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.850065 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-776ac1-openshift-default-dockercfg-fdxwz\"" Jul 15 14:27:14.858424 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.858395 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s"] Jul 15 14:27:14.949314 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-certs\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949357 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf6cd\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-kube-api-access-pf6cd\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949395 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-socket\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949458 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bb6761-0190-423e-8288-b79a610ad499-istiod-ca-cert\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949488 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bb6761-0190-423e-8288-b79a610ad499-istio-podinfo\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949504 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-credential-socket\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949789 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949535 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-envoy\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949789 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-data\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:14.949789 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:14.949628 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-istio-token\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050375 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-envoy\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050418 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-data\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050448 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-istio-token\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050480 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-certs\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050552 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf6cd\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-kube-api-access-pf6cd\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050580 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-socket\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050870 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050645 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bb6761-0190-423e-8288-b79a610ad499-istiod-ca-cert\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050870 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050679 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bb6761-0190-423e-8288-b79a610ad499-istio-podinfo\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050980 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050895 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-certs\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.050980 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050949 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-credential-socket\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.051082 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.050984 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-socket\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.051362 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.051341 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-credential-socket\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.051450 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.051419 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bb6761-0190-423e-8288-b79a610ad499-istiod-ca-cert\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.051450 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.051431 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-data\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.052869 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.052847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-envoy\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.053041 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.053023 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bb6761-0190-423e-8288-b79a610ad499-istio-podinfo\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.062382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.062362 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-istio-token\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.064178 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.064161 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf6cd\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-kube-api-access-pf6cd\") pod \"e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.160730 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.160630 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:15.311228 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.311191 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s"] Jul 15 14:27:15.315346 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:27:15.315313 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bb6761_0190_423e_8288_b79a610ad499.slice/crio-dafb5a57dda2827e5bc1461bafe50bbfcbca215b51a62c8cdd618a29f577c998 WatchSource:0}: Error finding container dafb5a57dda2827e5bc1461bafe50bbfcbca215b51a62c8cdd618a29f577c998: Status 404 returned error can't find the container with id dafb5a57dda2827e5bc1461bafe50bbfcbca215b51a62c8cdd618a29f577c998 Jul 15 14:27:15.317421 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.317385 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:15.317525 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.317468 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:15.317525 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:15.317511 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:16.114178 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:16.114134 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" event={"ID":"73bb6761-0190-423e-8288-b79a610ad499","Type":"ContainerStarted","Data":"0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818"} Jul 15 14:27:16.114667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:16.114195 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" event={"ID":"73bb6761-0190-423e-8288-b79a610ad499","Type":"ContainerStarted","Data":"dafb5a57dda2827e5bc1461bafe50bbfcbca215b51a62c8cdd618a29f577c998"} Jul 15 14:27:16.140430 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:16.140375 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podStartSLOduration=2.140352701 podStartE2EDuration="2.140352701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:27:16.137990651 +0000 UTC m=+2174.575599466" watchObservedRunningTime="2026-07-15 14:27:16.140352701 +0000 UTC m=+2174.577961494" Jul 15 14:27:16.161791 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:16.161763 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:16.166588 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:16.166568 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:17.120048 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:17.120017 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:17.121136 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:17.121115 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:23.425446 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.425412 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr"] Jul 15 14:27:23.429322 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.429278 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.432254 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.432224 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1bc685-openshift-default-dockercfg-9c7z6\"" Jul 15 14:27:23.443949 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.443924 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr"] Jul 15 14:27:23.534281 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534241 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwsrz\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-kube-api-access-fwsrz\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534485 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534380 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-data\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534485 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534433 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110e9bd7-2081-4483-84b3-bae70fcc5821-istiod-ca-cert\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534577 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534513 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-certs\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534577 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534561 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-podinfo\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534655 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534599 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-credential-socket\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534655 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534649 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-socket\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-token\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.534752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.534740 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-envoy\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635211 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwsrz\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-kube-api-access-fwsrz\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635502 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635265 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-data\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635502 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635316 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110e9bd7-2081-4483-84b3-bae70fcc5821-istiod-ca-cert\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635502 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635345 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-certs\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635502 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635374 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-podinfo\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635744 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-credential-socket\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635744 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635569 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-socket\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635744 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-token\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635744 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635648 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-envoy\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.635744 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635725 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-data\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.636000 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.635746 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-certs\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.636190 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.636161 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-credential-socket\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.636347 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.636328 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-socket\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.636547 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.636527 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110e9bd7-2081-4483-84b3-bae70fcc5821-istiod-ca-cert\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.637961 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.637937 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-envoy\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.638160 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.638140 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-podinfo\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.644732 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.644711 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-token\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.644818 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.644727 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwsrz\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-kube-api-access-fwsrz\") pod \"e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.741679 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.741596 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:23.901469 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.901444 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr"] Jul 15 14:27:23.903658 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:27:23.903629 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110e9bd7_2081_4483_84b3_bae70fcc5821.slice/crio-8bb928e79d88cb08644762966d0521da10a5bc74785cc3a887ac5cca9ef75f63 WatchSource:0}: Error finding container 8bb928e79d88cb08644762966d0521da10a5bc74785cc3a887ac5cca9ef75f63: Status 404 returned error can't find the container with id 8bb928e79d88cb08644762966d0521da10a5bc74785cc3a887ac5cca9ef75f63 Jul 15 14:27:23.905797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.905765 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:23.905861 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.905830 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:23.905899 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:23.905862 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:24.160561 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:24.160473 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" event={"ID":"110e9bd7-2081-4483-84b3-bae70fcc5821","Type":"ContainerStarted","Data":"5e846f8fb77159b369aebd1f9d3dd67c1adef4bf7eb81420c698a5b9a76b8193"} Jul 15 14:27:24.160561 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:24.160537 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" event={"ID":"110e9bd7-2081-4483-84b3-bae70fcc5821","Type":"ContainerStarted","Data":"8bb928e79d88cb08644762966d0521da10a5bc74785cc3a887ac5cca9ef75f63"} Jul 15 14:27:24.186014 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:24.185960 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podStartSLOduration=1.185942479 podStartE2EDuration="1.185942479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:27:24.183568358 +0000 UTC m=+2182.621177150" watchObservedRunningTime="2026-07-15 14:27:24.185942479 +0000 UTC m=+2182.623551270" Jul 15 14:27:24.742758 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:24.742704 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:24.748133 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:24.748105 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:25.166158 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:25.166126 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:25.167064 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:25.167047 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:33.162459 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:33.162425 2584 patch_prober.go:28] interesting pod/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 14:27:33.162905 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:33.162510 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 14:27:34.219726 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:34.219637 2584 generic.go:356] "Generic (PLEG): container finished" podID="73bb6761-0190-423e-8288-b79a610ad499" containerID="0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818" exitCode=0 Jul 15 14:27:34.219726 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:34.219711 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" event={"ID":"73bb6761-0190-423e-8288-b79a610ad499","Type":"ContainerDied","Data":"0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818"} Jul 15 14:27:34.220182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:34.220116 2584 scope.go:117] "RemoveContainer" containerID="0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818" Jul 15 14:27:34.221115 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:34.221086 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:34.221219 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:34.221166 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:34.221281 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:34.221217 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:35.227584 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:35.227550 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" event={"ID":"73bb6761-0190-423e-8288-b79a610ad499","Type":"ContainerStarted","Data":"810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a"} Jul 15 14:27:36.161699 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:36.161663 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:36.161990 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:36.161718 2584 patch_prober.go:28] interesting pod/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 15 14:27:36.162170 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:36.162146 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 15 14:27:37.161728 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:37.161675 2584 patch_prober.go:28] interesting pod/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 15 14:27:37.162251 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:37.161770 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 15 14:27:38.161321 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:38.161260 2584 patch_prober.go:28] interesting pod/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 15 14:27:38.161653 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:38.161622 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 15 14:27:39.162032 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:39.161990 2584 patch_prober.go:28] interesting pod/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 15 14:27:39.162659 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:39.162625 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 15 14:27:40.165773 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:40.165737 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:40.166178 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:40.166103 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:40.167037 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:40.167017 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:27:41.208468 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:41.208432 2584 patch_prober.go:28] interesting pod/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 14:27:41.208844 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:41.208496 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 14:27:42.278061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:42.277961 2584 generic.go:356] "Generic (PLEG): container finished" podID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerID="5e846f8fb77159b369aebd1f9d3dd67c1adef4bf7eb81420c698a5b9a76b8193" exitCode=0 Jul 15 14:27:42.278061 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:42.278036 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" event={"ID":"110e9bd7-2081-4483-84b3-bae70fcc5821","Type":"ContainerDied","Data":"5e846f8fb77159b369aebd1f9d3dd67c1adef4bf7eb81420c698a5b9a76b8193"} Jul 15 14:27:42.278520 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:42.278493 2584 scope.go:117] "RemoveContainer" containerID="5e846f8fb77159b369aebd1f9d3dd67c1adef4bf7eb81420c698a5b9a76b8193" Jul 15 14:27:42.279535 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:42.279506 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:42.279612 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:42.279574 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:42.279612 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:42.279604 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:27:43.285265 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:43.285234 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" event={"ID":"110e9bd7-2081-4483-84b3-bae70fcc5821","Type":"ContainerStarted","Data":"fe5767cc6dc4dffb70b256830a2a5149769bbedf9cd2f1cb60f05aac3e2f12bb"} Jul 15 14:27:43.742198 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:43.742164 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:43.742578 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:43.742556 2584 patch_prober.go:28] interesting pod/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 15 14:27:43.742744 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:43.742719 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 15 14:27:44.742608 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:44.742563 2584 patch_prober.go:28] interesting pod/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 15 14:27:44.743148 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:44.742635 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 15 14:27:45.742975 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:45.742936 2584 patch_prober.go:28] interesting pod/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 15 14:27:45.743618 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:45.743585 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 15 14:27:46.742389 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:46.742348 2584 patch_prober.go:28] interesting pod/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 15 14:27:46.742676 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:46.742646 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 15 14:27:47.745868 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:47.745835 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:47.746282 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:47.746179 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:27:47.746804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:27:47.746785 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:29:07.243085 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:07.243042 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s"] Jul 15 14:29:07.243752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:07.243347 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" containerID="cri-o://810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a" gracePeriod=30 Jul 15 14:29:10.169269 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:10.169230 2584 patch_prober.go:28] interesting pod/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 14:29:10.169667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:10.169286 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 14:29:12.507596 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.507536 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:29:12.623186 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623159 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-data\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623257 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf6cd\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-kube-api-access-pf6cd\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623281 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-istio-token\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623338 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-credential-socket\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623367 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-certs\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623606 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623398 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-socket\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623606 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623441 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bb6761-0190-423e-8288-b79a610ad499-istio-podinfo\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623606 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623476 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bb6761-0190-423e-8288-b79a610ad499-istiod-ca-cert\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623606 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623511 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-envoy\") pod \"73bb6761-0190-423e-8288-b79a610ad499\" (UID: \"73bb6761-0190-423e-8288-b79a610ad499\") " Jul 15 14:29:12.623811 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623687 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:12.623811 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623696 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:12.623811 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623710 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:12.623973 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623890 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.623973 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623909 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.623973 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.623924 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.624115 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.624018 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bb6761-0190-423e-8288-b79a610ad499-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:29:12.624163 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.624125 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-data" (OuterVolumeSpecName: "istio-data") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:12.625591 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.625566 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73bb6761-0190-423e-8288-b79a610ad499-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:29:12.625710 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.625689 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-istio-token" (OuterVolumeSpecName: "istio-token") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:29:12.625828 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.625813 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-kube-api-access-pf6cd" (OuterVolumeSpecName: "kube-api-access-pf6cd") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "kube-api-access-pf6cd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:29:12.625957 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.625930 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73bb6761-0190-423e-8288-b79a610ad499" (UID: "73bb6761-0190-423e-8288-b79a610ad499"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:12.725384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.725341 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf6cd\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-kube-api-access-pf6cd\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.725384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.725378 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bb6761-0190-423e-8288-b79a610ad499-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.725384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.725388 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bb6761-0190-423e-8288-b79a610ad499-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.725625 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.725396 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bb6761-0190-423e-8288-b79a610ad499-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.725625 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.725404 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.725625 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.725412 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bb6761-0190-423e-8288-b79a610ad499-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:12.767371 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.767262 2584 generic.go:356] "Generic (PLEG): container finished" podID="73bb6761-0190-423e-8288-b79a610ad499" containerID="810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a" exitCode=0 Jul 15 14:29:12.767371 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.767335 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" Jul 15 14:29:12.767371 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.767356 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" event={"ID":"73bb6761-0190-423e-8288-b79a610ad499","Type":"ContainerDied","Data":"810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a"} Jul 15 14:29:12.767643 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.767424 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s" event={"ID":"73bb6761-0190-423e-8288-b79a610ad499","Type":"ContainerDied","Data":"dafb5a57dda2827e5bc1461bafe50bbfcbca215b51a62c8cdd618a29f577c998"} Jul 15 14:29:12.767643 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.767458 2584 scope.go:117] "RemoveContainer" containerID="810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a" Jul 15 14:29:12.776661 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.776646 2584 scope.go:117] "RemoveContainer" containerID="0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818" Jul 15 14:29:12.784906 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.784889 2584 scope.go:117] "RemoveContainer" containerID="810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a" Jul 15 14:29:12.785146 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:29:12.785129 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a\": container with ID starting with 810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a not found: ID does not exist" containerID="810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a" Jul 15 14:29:12.785202 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.785153 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a"} err="failed to get container status \"810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a\": rpc error: code = NotFound desc = could not find container \"810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a\": container with ID starting with 810ae0f3b35aac9c654e20ad56fa38cbc4ba9a1689212cbf41cc28355f44471a not found: ID does not exist" Jul 15 14:29:12.785202 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.785169 2584 scope.go:117] "RemoveContainer" containerID="0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818" Jul 15 14:29:12.785388 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:29:12.785371 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818\": container with ID starting with 0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818 not found: ID does not exist" containerID="0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818" Jul 15 14:29:12.785435 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.785392 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818"} err="failed to get container status \"0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818\": rpc error: code = NotFound desc = could not find container \"0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818\": container with ID starting with 0e6e1dca792a40b46b920f79f2751d92b4b6787e7a9d077dfda2243282b92818 not found: ID does not exist" Jul 15 14:29:12.793193 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.793170 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s"] Jul 15 14:29:12.801393 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:12.801373 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-776ac1-openshift-default-65fb5bf55-xhp9s"] Jul 15 14:29:14.008230 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:14.008196 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73bb6761-0190-423e-8288-b79a610ad499" path="/var/lib/kubelet/pods/73bb6761-0190-423e-8288-b79a610ad499/volumes" Jul 15 14:29:33.794959 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:33.794924 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr"] Jul 15 14:29:33.795392 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:33.795129 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" containerID="cri-o://fe5767cc6dc4dffb70b256830a2a5149769bbedf9cd2f1cb60f05aac3e2f12bb" gracePeriod=30 Jul 15 14:29:35.215566 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.215532 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4"] Jul 15 14:29:35.215937 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.215927 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" Jul 15 14:29:35.215980 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.215939 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" Jul 15 14:29:35.216015 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.216011 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" Jul 15 14:29:35.216049 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.216021 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" Jul 15 14:29:35.220438 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.220419 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.261197 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.261170 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-752a1c-openshift-default-dockercfg-rjx7m\"" Jul 15 14:29:35.320849 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.320805 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-data\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.320851 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-podinfo\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.320901 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddvn7\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-kube-api-access-ddvn7\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.320929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-certs\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.320946 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-token\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.320971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-envoy\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.321025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae3e209-4e73-47da-a070-3fd389c6be14-istiod-ca-cert\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321426 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.321052 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-credential-socket\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.321426 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.321072 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-socket\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.421605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421572 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddvn7\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-kube-api-access-ddvn7\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.421784 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421611 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-certs\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.421784 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421638 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-token\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.421784 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421679 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-envoy\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.421784 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421726 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae3e209-4e73-47da-a070-3fd389c6be14-istiod-ca-cert\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.421784 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421771 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-credential-socket\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422050 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421799 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-socket\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422050 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421881 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-data\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422050 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.421912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-podinfo\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422213 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.422097 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-certs\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422270 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.422253 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-credential-socket\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.422341 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-data\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422521 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.422495 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-socket\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.422653 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.422591 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae3e209-4e73-47da-a070-3fd389c6be14-istiod-ca-cert\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.424103 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.424078 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-envoy\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.424470 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.424451 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-podinfo\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.466430 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.466347 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddvn7\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-kube-api-access-ddvn7\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.470387 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.470359 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-token\") pod \"e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.527439 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.527408 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4"] Jul 15 14:29:35.530336 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.530284 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:35.700980 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.700957 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4"] Jul 15 14:29:35.702627 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:29:35.702601 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae3e209_4e73_47da_a070_3fd389c6be14.slice/crio-949f6e382dd7275d9583d8a61b524b17950453e741dc210bc6aea60f00c18231 WatchSource:0}: Error finding container 949f6e382dd7275d9583d8a61b524b17950453e741dc210bc6aea60f00c18231: Status 404 returned error can't find the container with id 949f6e382dd7275d9583d8a61b524b17950453e741dc210bc6aea60f00c18231 Jul 15 14:29:35.704665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.704631 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:35.704734 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.704702 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:35.704734 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.704730 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:35.892186 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.892133 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" event={"ID":"7ae3e209-4e73-47da-a070-3fd389c6be14","Type":"ContainerStarted","Data":"a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8"} Jul 15 14:29:35.892410 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.892207 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" event={"ID":"7ae3e209-4e73-47da-a070-3fd389c6be14","Type":"ContainerStarted","Data":"949f6e382dd7275d9583d8a61b524b17950453e741dc210bc6aea60f00c18231"} Jul 15 14:29:35.942511 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:35.942458 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podStartSLOduration=0.942436658 podStartE2EDuration="942.436658ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:29:35.940085531 +0000 UTC m=+2314.377694338" watchObservedRunningTime="2026-07-15 14:29:35.942436658 +0000 UTC m=+2314.380045451" Jul 15 14:29:36.531405 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:36.531357 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:36.537870 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:36.537842 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:36.897621 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:36.897591 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:36.898459 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:36.898438 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:38.914327 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:38.914280 2584 generic.go:356] "Generic (PLEG): container finished" podID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerID="fe5767cc6dc4dffb70b256830a2a5149769bbedf9cd2f1cb60f05aac3e2f12bb" exitCode=0 Jul 15 14:29:38.914762 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:38.914332 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" event={"ID":"110e9bd7-2081-4483-84b3-bae70fcc5821","Type":"ContainerDied","Data":"fe5767cc6dc4dffb70b256830a2a5149769bbedf9cd2f1cb60f05aac3e2f12bb"} Jul 15 14:29:38.914762 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:38.914412 2584 scope.go:117] "RemoveContainer" containerID="5e846f8fb77159b369aebd1f9d3dd67c1adef4bf7eb81420c698a5b9a76b8193" Jul 15 14:29:39.049279 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.049256 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:29:39.158689 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.158659 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-data\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.158888 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.158752 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-token\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.158888 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.158782 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110e9bd7-2081-4483-84b3-bae70fcc5821-istiod-ca-cert\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.158888 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.158833 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-certs\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.158888 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.158876 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwsrz\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-kube-api-access-fwsrz\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159138 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-envoy\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159195 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/110e9bd7-2081-4483-84b3-bae70fcc5821-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159237 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-credential-socket\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159580 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159613 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-socket\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159648 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-podinfo\") pod \"110e9bd7-2081-4483-84b3-bae70fcc5821\" (UID: \"110e9bd7-2081-4483-84b3-bae70fcc5821\") " Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159653 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-data" (OuterVolumeSpecName: "istio-data") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159875 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:39.159941 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.159535 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:39.160605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.160165 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.160605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.160189 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.160605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.160203 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.160605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.160216 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110e9bd7-2081-4483-84b3-bae70fcc5821-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.160605 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.160245 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.161360 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.161321 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-token" (OuterVolumeSpecName: "istio-token") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:29:39.161508 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.161442 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-kube-api-access-fwsrz" (OuterVolumeSpecName: "kube-api-access-fwsrz") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "kube-api-access-fwsrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:29:39.161769 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.161747 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:29:39.161894 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.161877 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "110e9bd7-2081-4483-84b3-bae70fcc5821" (UID: "110e9bd7-2081-4483-84b3-bae70fcc5821"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:29:39.260804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.260777 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.260804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.260801 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.260804 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.260811 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwsrz\" (UniqueName: \"kubernetes.io/projected/110e9bd7-2081-4483-84b3-bae70fcc5821-kube-api-access-fwsrz\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.261024 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.260820 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110e9bd7-2081-4483-84b3-bae70fcc5821-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:29:39.920644 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.920594 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" event={"ID":"110e9bd7-2081-4483-84b3-bae70fcc5821","Type":"ContainerDied","Data":"8bb928e79d88cb08644762966d0521da10a5bc74785cc3a887ac5cca9ef75f63"} Jul 15 14:29:39.920644 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.920629 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr" Jul 15 14:29:39.921153 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.920669 2584 scope.go:117] "RemoveContainer" containerID="fe5767cc6dc4dffb70b256830a2a5149769bbedf9cd2f1cb60f05aac3e2f12bb" Jul 15 14:29:39.947752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.947727 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr"] Jul 15 14:29:39.951217 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:39.951195 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc685-openshift-default-66b58f675c-gdkkr"] Jul 15 14:29:40.009128 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:40.009097 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" path="/var/lib/kubelet/pods/110e9bd7-2081-4483-84b3-bae70fcc5821/volumes" Jul 15 14:29:43.335731 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.335696 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7"] Jul 15 14:29:43.336176 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336144 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" Jul 15 14:29:43.336176 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336158 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bb6761-0190-423e-8288-b79a610ad499" containerName="istio-proxy" Jul 15 14:29:43.336176 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336178 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" Jul 15 14:29:43.336365 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336185 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" Jul 15 14:29:43.336365 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336190 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" Jul 15 14:29:43.336365 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336196 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" Jul 15 14:29:43.336365 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336253 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" Jul 15 14:29:43.336365 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.336260 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="110e9bd7-2081-4483-84b3-bae70fcc5821" containerName="istio-proxy" Jul 15 14:29:43.340702 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.340680 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.348708 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.348692 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e7e87a-openshift-default-dockercfg-wp9wb\"" Jul 15 14:29:43.356594 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.356572 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7"] Jul 15 14:29:43.400819 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.400792 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-data\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.400938 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.400832 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb179d4-107d-4819-872e-afd2ab624097-istiod-ca-cert\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.400988 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.400923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-envoy\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.400988 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.400980 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-credential-socket\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.401060 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.401010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74fp\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-kube-api-access-w74fp\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.401060 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.401040 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb179d4-107d-4819-872e-afd2ab624097-istio-podinfo\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.401060 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.401058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-socket\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.401168 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.401091 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-certs\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.401168 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.401108 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-istio-token\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502131 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502095 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-certs\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502131 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502130 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-istio-token\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502344 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-data\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502344 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502222 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb179d4-107d-4819-872e-afd2ab624097-istiod-ca-cert\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502344 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502287 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-envoy\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502475 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-credential-socket\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502475 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502389 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74fp\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-kube-api-access-w74fp\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502475 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502427 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb179d4-107d-4819-872e-afd2ab624097-istio-podinfo\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502632 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502558 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-socket\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502693 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502645 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-data\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502809 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502786 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-credential-socket\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502891 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502835 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-certs\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502891 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502873 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-socket\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.502988 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.502961 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb179d4-107d-4819-872e-afd2ab624097-istiod-ca-cert\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.504538 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.504516 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-envoy\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.504752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.504734 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb179d4-107d-4819-872e-afd2ab624097-istio-podinfo\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.511247 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.511225 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-istio-token\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.511650 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.511631 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74fp\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-kube-api-access-w74fp\") pod \"e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.651044 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.651010 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:43.803367 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.803339 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7"] Jul 15 14:29:43.804931 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:29:43.804903 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb179d4_107d_4819_872e_afd2ab624097.slice/crio-e12f0d35037f2876ddeed73f576ffd21cab71c9ef4300aef7ed218e8e2fa63f6 WatchSource:0}: Error finding container e12f0d35037f2876ddeed73f576ffd21cab71c9ef4300aef7ed218e8e2fa63f6: Status 404 returned error can't find the container with id e12f0d35037f2876ddeed73f576ffd21cab71c9ef4300aef7ed218e8e2fa63f6 Jul 15 14:29:43.807047 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.807014 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:43.807162 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.807094 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:43.807162 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.807142 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:43.945571 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.945471 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" event={"ID":"1cb179d4-107d-4819-872e-afd2ab624097","Type":"ContainerStarted","Data":"ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893"} Jul 15 14:29:43.945571 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.945532 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" event={"ID":"1cb179d4-107d-4819-872e-afd2ab624097","Type":"ContainerStarted","Data":"e12f0d35037f2876ddeed73f576ffd21cab71c9ef4300aef7ed218e8e2fa63f6"} Jul 15 14:29:43.982333 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:43.980041 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" podStartSLOduration=0.980016915 podStartE2EDuration="980.016915ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:29:43.978057943 +0000 UTC m=+2322.415666768" watchObservedRunningTime="2026-07-15 14:29:43.980016915 +0000 UTC m=+2322.417625709" Jul 15 14:29:44.651809 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:44.651767 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:44.660107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:44.659847 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:44.953359 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:44.953240 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:44.959858 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:44.959824 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:29:51.898680 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.898640 2584 patch_prober.go:28] interesting pod/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 15 14:29:51.899096 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.898698 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 15 14:29:51.992626 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.992595 2584 generic.go:356] "Generic (PLEG): container finished" podID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerID="a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8" exitCode=0 Jul 15 14:29:51.992791 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.992671 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" event={"ID":"7ae3e209-4e73-47da-a070-3fd389c6be14","Type":"ContainerDied","Data":"a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8"} Jul 15 14:29:51.993097 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.993083 2584 scope.go:117] "RemoveContainer" containerID="a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8" Jul 15 14:29:51.994012 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.993985 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:51.994066 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.994055 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:51.994111 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:51.994088 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:52.999525 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:52.999483 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" event={"ID":"7ae3e209-4e73-47da-a070-3fd389c6be14","Type":"ContainerStarted","Data":"ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236"} Jul 15 14:29:53.530879 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:53.530830 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:53.531361 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:53.531329 2584 patch_prober.go:28] interesting pod/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 15 14:29:53.531506 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:53.531417 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 15 14:29:54.531777 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:54.531683 2584 patch_prober.go:28] interesting pod/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 15 14:29:54.532452 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:54.532413 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 15 14:29:55.531769 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:55.531736 2584 patch_prober.go:28] interesting pod/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 15 14:29:55.532035 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:55.532007 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 15 14:29:56.531394 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:56.531349 2584 patch_prober.go:28] interesting pod/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 15 14:29:56.531593 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:56.531434 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 15 14:29:57.535427 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:57.535385 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:57.535827 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:57.535802 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:57.536422 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:57.536398 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:29:59.033119 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.033085 2584 generic.go:356] "Generic (PLEG): container finished" podID="1cb179d4-107d-4819-872e-afd2ab624097" containerID="ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893" exitCode=0 Jul 15 14:29:59.033590 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.033160 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" event={"ID":"1cb179d4-107d-4819-872e-afd2ab624097","Type":"ContainerDied","Data":"ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893"} Jul 15 14:29:59.033672 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.033616 2584 scope.go:117] "RemoveContainer" containerID="ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893" Jul 15 14:29:59.034671 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.034636 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:59.034795 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.034742 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:59.034795 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.034786 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:29:59.953104 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:29:59.953070 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:00.041045 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:00.041012 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" event={"ID":"1cb179d4-107d-4819-872e-afd2ab624097","Type":"ContainerStarted","Data":"d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9"} Jul 15 14:30:00.652322 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:00.652260 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:00.653710 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:00.653679 2584 patch_prober.go:28] interesting pod/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 15 14:30:00.653836 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:00.653741 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 15 14:30:01.651702 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:01.651653 2584 patch_prober.go:28] interesting pod/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 15 14:30:01.652216 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:01.651724 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 15 14:30:02.652647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:02.652611 2584 patch_prober.go:28] interesting pod/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 15 14:30:02.653222 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:02.653191 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 15 14:30:03.652005 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:03.651972 2584 patch_prober.go:28] interesting pod/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 15 14:30:03.652763 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:03.652263 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 15 14:30:04.655312 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:04.655274 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:04.655773 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:04.655607 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:04.656176 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:04.656158 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:24.599188 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:24.599154 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4"] Jul 15 14:30:24.599780 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:24.599367 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" containerID="cri-o://ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236" gracePeriod=30 Jul 15 14:30:27.538602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:27.538561 2584 patch_prober.go:28] interesting pod/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 14:30:27.539046 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:27.538634 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 14:30:29.862647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.862585 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:30:29.988567 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988538 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-certs\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988586 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-data\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988609 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddvn7\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-kube-api-access-ddvn7\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988652 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae3e209-4e73-47da-a070-3fd389c6be14-istiod-ca-cert\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988710 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-token\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988739 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-envoy\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988997 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988820 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-socket\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988997 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988863 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-credential-socket\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988997 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988890 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-podinfo\") pod \"7ae3e209-4e73-47da-a070-3fd389c6be14\" (UID: \"7ae3e209-4e73-47da-a070-3fd389c6be14\") " Jul 15 14:30:29.988997 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.988905 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:29.989589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989060 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:29.989589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989158 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:29.989589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989222 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:29.989589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989239 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:29.989589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989257 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:29.989589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989519 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ae3e209-4e73-47da-a070-3fd389c6be14-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:30:29.989918 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.989620 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:29.990973 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.990948 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-kube-api-access-ddvn7" (OuterVolumeSpecName: "kube-api-access-ddvn7") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "kube-api-access-ddvn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:30:29.991094 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.991078 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:30:29.991396 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.991382 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:30:29.991449 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:29.991409 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ae3e209-4e73-47da-a070-3fd389c6be14" (UID: "7ae3e209-4e73-47da-a070-3fd389c6be14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:30.090226 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.090203 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:30.090226 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.090225 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:30.090390 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.090234 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:30.090390 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.090242 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ae3e209-4e73-47da-a070-3fd389c6be14-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:30.090390 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.090251 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddvn7\" (UniqueName: \"kubernetes.io/projected/7ae3e209-4e73-47da-a070-3fd389c6be14-kube-api-access-ddvn7\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:30.090390 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.090259 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ae3e209-4e73-47da-a070-3fd389c6be14-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:30.204758 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.204667 2584 generic.go:356] "Generic (PLEG): container finished" podID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerID="ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236" exitCode=0 Jul 15 14:30:30.204758 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.204711 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" Jul 15 14:30:30.204758 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.204724 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" event={"ID":"7ae3e209-4e73-47da-a070-3fd389c6be14","Type":"ContainerDied","Data":"ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236"} Jul 15 14:30:30.205001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.204777 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4" event={"ID":"7ae3e209-4e73-47da-a070-3fd389c6be14","Type":"ContainerDied","Data":"949f6e382dd7275d9583d8a61b524b17950453e741dc210bc6aea60f00c18231"} Jul 15 14:30:30.205001 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.204804 2584 scope.go:117] "RemoveContainer" containerID="ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236" Jul 15 14:30:30.215857 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.215841 2584 scope.go:117] "RemoveContainer" containerID="a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8" Jul 15 14:30:30.222640 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.222619 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4"] Jul 15 14:30:30.224256 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.224239 2584 scope.go:117] "RemoveContainer" containerID="ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236" Jul 15 14:30:30.224538 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:30:30.224517 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236\": container with ID starting with ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236 not found: ID does not exist" containerID="ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236" Jul 15 14:30:30.224618 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.224544 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236"} err="failed to get container status \"ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236\": rpc error: code = NotFound desc = could not find container \"ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236\": container with ID starting with ba0dbfcc26fe7acb187502dc7a4fbadcdba417035812b1630cee834912c89236 not found: ID does not exist" Jul 15 14:30:30.224618 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.224566 2584 scope.go:117] "RemoveContainer" containerID="a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8" Jul 15 14:30:30.224816 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:30:30.224796 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8\": container with ID starting with a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8 not found: ID does not exist" containerID="a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8" Jul 15 14:30:30.224875 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.224825 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8"} err="failed to get container status \"a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8\": rpc error: code = NotFound desc = could not find container \"a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8\": container with ID starting with a10977105f9877516e1637aa634236cbfba5d6b100f121ed2ea3ecb68b7e01f8 not found: ID does not exist" Jul 15 14:30:30.226120 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:30.226103 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-752a1c-openshift-default-5745745b9c-m7fc4"] Jul 15 14:30:32.008147 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:32.008114 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" path="/var/lib/kubelet/pods/7ae3e209-4e73-47da-a070-3fd389c6be14/volumes" Jul 15 14:30:50.824931 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:50.824900 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7"] Jul 15 14:30:50.825475 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:50.825107 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" containerID="cri-o://d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9" gracePeriod=30 Jul 15 14:30:51.752969 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.752931 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px"] Jul 15 14:30:51.753376 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.753362 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" Jul 15 14:30:51.753376 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.753377 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" Jul 15 14:30:51.753480 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.753394 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" Jul 15 14:30:51.753480 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.753401 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" Jul 15 14:30:51.753480 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.753477 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" Jul 15 14:30:51.756703 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.756684 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.759707 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.759689 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0b4a59-openshift-default-dockercfg-vzvcs\"" Jul 15 14:30:51.768844 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.768823 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px"] Jul 15 14:30:51.793193 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793166 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-certs\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793353 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793207 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-podinfo\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793353 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793226 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-credential-socket\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793353 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793272 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-envoy\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793353 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793337 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-data\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793508 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793359 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-token\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793508 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793461 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-socket\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793508 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793490 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47ttg\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-kube-api-access-47ttg\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.793602 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.793557 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istiod-ca-cert\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.894777 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894746 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-socket\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894788 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47ttg\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-kube-api-access-47ttg\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894835 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istiod-ca-cert\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894863 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-certs\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894884 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-podinfo\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894902 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-credential-socket\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894920 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-envoy\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894943 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-data\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895246 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.894967 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-token\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.895265 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-certs\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.895378 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-credential-socket\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.895450 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-data\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.895533 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-socket\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.895667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.895633 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istiod-ca-cert\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.897208 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.897186 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-envoy\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.897478 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.897462 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-podinfo\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.903861 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.903837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-token\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:51.904094 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:51.904074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47ttg\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-kube-api-access-47ttg\") pod \"e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:52.066558 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.066462 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:52.222860 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.222793 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:30:52.223031 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.222887 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:30:52.223031 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.222928 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:30:52.223981 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.223954 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px"] Jul 15 14:30:52.325308 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.325193 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" event={"ID":"d1671de3-2b58-4dc2-bfed-9d7467ba68aa","Type":"ContainerStarted","Data":"bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77"} Jul 15 14:30:52.325308 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.325257 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" event={"ID":"d1671de3-2b58-4dc2-bfed-9d7467ba68aa","Type":"ContainerStarted","Data":"fcf26727e23c82d2e796f0dbd3f02fa9c9b7b327d344a35d678eca1330d6c60c"} Jul 15 14:30:52.347633 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:52.347571 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podStartSLOduration=1.347548251 podStartE2EDuration="1.347548251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:30:52.344746767 +0000 UTC m=+2390.782355559" watchObservedRunningTime="2026-07-15 14:30:52.347548251 +0000 UTC m=+2390.785157045" Jul 15 14:30:53.067464 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:53.067427 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:53.072967 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:53.072943 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:53.330639 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:53.330550 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:53.331866 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:53.331838 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:30:56.098492 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.098415 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:56.241424 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241386 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-istio-token\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241424 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241426 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb179d4-107d-4819-872e-afd2ab624097-istiod-ca-cert\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241682 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241468 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-socket\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241682 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241508 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb179d4-107d-4819-872e-afd2ab624097-istio-podinfo\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241682 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241586 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-data\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241682 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241638 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-credential-socket\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241682 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241682 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-certs\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241958 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241710 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w74fp\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-kube-api-access-w74fp\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241958 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241748 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-envoy\") pod \"1cb179d4-107d-4819-872e-afd2ab624097\" (UID: \"1cb179d4-107d-4819-872e-afd2ab624097\") " Jul 15 14:30:56.241958 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241838 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cb179d4-107d-4819-872e-afd2ab624097-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:30:56.241958 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.241856 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:56.242180 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.242121 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb179d4-107d-4819-872e-afd2ab624097-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.242180 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.242139 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.242280 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.242249 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:56.242356 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.242332 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:56.242885 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.242859 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:56.243934 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.243910 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:30:56.244013 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.243979 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cb179d4-107d-4819-872e-afd2ab624097-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:30:56.244256 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.244234 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-kube-api-access-w74fp" (OuterVolumeSpecName: "kube-api-access-w74fp") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "kube-api-access-w74fp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:30:56.244384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.244319 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cb179d4-107d-4819-872e-afd2ab624097" (UID: "1cb179d4-107d-4819-872e-afd2ab624097"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:30:56.343645 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343618 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.343645 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343640 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb179d4-107d-4819-872e-afd2ab624097-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.343645 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343648 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.343859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343656 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.343859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343664 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.343859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343673 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w74fp\" (UniqueName: \"kubernetes.io/projected/1cb179d4-107d-4819-872e-afd2ab624097-kube-api-access-w74fp\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.343859 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.343681 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb179d4-107d-4819-872e-afd2ab624097-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:30:56.350766 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.350689 2584 generic.go:356] "Generic (PLEG): container finished" podID="1cb179d4-107d-4819-872e-afd2ab624097" containerID="d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9" exitCode=0 Jul 15 14:30:56.350766 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.350761 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" Jul 15 14:30:56.350936 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.350771 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" event={"ID":"1cb179d4-107d-4819-872e-afd2ab624097","Type":"ContainerDied","Data":"d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9"} Jul 15 14:30:56.350936 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.350825 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7" event={"ID":"1cb179d4-107d-4819-872e-afd2ab624097","Type":"ContainerDied","Data":"e12f0d35037f2876ddeed73f576ffd21cab71c9ef4300aef7ed218e8e2fa63f6"} Jul 15 14:30:56.350936 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.350846 2584 scope.go:117] "RemoveContainer" containerID="d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9" Jul 15 14:30:56.360809 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.360793 2584 scope.go:117] "RemoveContainer" containerID="ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893" Jul 15 14:30:56.370053 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.370035 2584 scope.go:117] "RemoveContainer" containerID="d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9" Jul 15 14:30:56.370341 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:30:56.370322 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9\": container with ID starting with d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9 not found: ID does not exist" containerID="d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9" Jul 15 14:30:56.370396 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.370351 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9"} err="failed to get container status \"d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9\": rpc error: code = NotFound desc = could not find container \"d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9\": container with ID starting with d73e43184ff0143ec5691a011c8693e34cc845fc89749fa342698b587ae4afc9 not found: ID does not exist" Jul 15 14:30:56.370396 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.370368 2584 scope.go:117] "RemoveContainer" containerID="ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893" Jul 15 14:30:56.370699 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:30:56.370659 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893\": container with ID starting with ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893 not found: ID does not exist" containerID="ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893" Jul 15 14:30:56.370699 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.370686 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893"} err="failed to get container status \"ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893\": rpc error: code = NotFound desc = could not find container \"ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893\": container with ID starting with ca02a41ab209b28c5f15c7d6b6410291ab199b9b623e24f26eb0888584d71893 not found: ID does not exist" Jul 15 14:30:56.373352 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.373329 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7"] Jul 15 14:30:56.379053 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:56.379034 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e7e87a-openshift-default-577fcccc47-5k8m7"] Jul 15 14:30:58.007991 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:58.007941 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cb179d4-107d-4819-872e-afd2ab624097" path="/var/lib/kubelet/pods/1cb179d4-107d-4819-872e-afd2ab624097/volumes" Jul 15 14:30:59.480799 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.480769 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758"] Jul 15 14:30:59.481268 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481175 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" Jul 15 14:30:59.481268 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481187 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" Jul 15 14:30:59.481268 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481199 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" Jul 15 14:30:59.481268 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481204 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" Jul 15 14:30:59.481268 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481263 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" Jul 15 14:30:59.481574 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481273 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cb179d4-107d-4819-872e-afd2ab624097" containerName="istio-proxy" Jul 15 14:30:59.481574 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.481283 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae3e209-4e73-47da-a070-3fd389c6be14" containerName="istio-proxy" Jul 15 14:30:59.485923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.485898 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.489385 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.489365 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0a4dfd-openshift-default-dockercfg-bkbgs\"" Jul 15 14:30:59.500462 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.500439 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758"] Jul 15 14:30:59.573212 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-data\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573212 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-envoy\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573496 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573244 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-credential-socket\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573496 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573371 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-socket\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573496 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573412 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-certs\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573496 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573445 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-podinfo\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573668 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573502 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26acd0a8-5843-4f8f-8c38-8b5419d58482-istiod-ca-cert\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573668 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573535 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-token\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.573668 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.573620 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pcvp\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-kube-api-access-8pcvp\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674461 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pcvp\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-kube-api-access-8pcvp\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674492 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-data\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-envoy\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674559 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-credential-socket\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674617 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-socket\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674646 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-certs\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674957 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674675 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-podinfo\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674957 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674803 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26acd0a8-5843-4f8f-8c38-8b5419d58482-istiod-ca-cert\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674957 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674843 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-token\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.674957 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674927 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-data\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.675156 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.674997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-certs\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.675156 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.675065 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-socket\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.675156 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.675128 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-credential-socket\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.675565 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.675546 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26acd0a8-5843-4f8f-8c38-8b5419d58482-istiod-ca-cert\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.676841 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.676822 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-envoy\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.677107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.677091 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-podinfo\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.683808 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.683779 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pcvp\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-kube-api-access-8pcvp\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.683906 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.683840 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-token\") pod \"e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.796908 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.796821 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:30:59.947110 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.947076 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758"] Jul 15 14:30:59.949499 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:30:59.949472 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26acd0a8_5843_4f8f_8c38_8b5419d58482.slice/crio-25c96b9fd867dabda34b25a04170711c984d99ddf0b51221fae459223d3bf24c WatchSource:0}: Error finding container 25c96b9fd867dabda34b25a04170711c984d99ddf0b51221fae459223d3bf24c: Status 404 returned error can't find the container with id 25c96b9fd867dabda34b25a04170711c984d99ddf0b51221fae459223d3bf24c Jul 15 14:30:59.951702 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.951668 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:30:59.951776 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.951738 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:30:59.951776 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:30:59.951768 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:00.374745 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:00.374701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" event={"ID":"26acd0a8-5843-4f8f-8c38-8b5419d58482","Type":"ContainerStarted","Data":"883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d"} Jul 15 14:31:00.374942 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:00.374770 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" event={"ID":"26acd0a8-5843-4f8f-8c38-8b5419d58482","Type":"ContainerStarted","Data":"25c96b9fd867dabda34b25a04170711c984d99ddf0b51221fae459223d3bf24c"} Jul 15 14:31:00.417357 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:00.417311 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" podStartSLOduration=1.417275945 podStartE2EDuration="1.417275945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:31:00.416372744 +0000 UTC m=+2398.853981535" watchObservedRunningTime="2026-07-15 14:31:00.417275945 +0000 UTC m=+2398.854884738" Jul 15 14:31:00.797346 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:00.797222 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:01.803350 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:01.803322 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:02.156022 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:02.155982 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:31:02.162782 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:02.162757 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:31:02.167771 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:02.167734 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:31:02.172721 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:02.172689 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:31:02.385745 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:02.385712 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:02.386854 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:02.386830 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:08.332829 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.332790 2584 patch_prober.go:28] interesting pod/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 15 14:31:08.333507 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.333473 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 15 14:31:08.417419 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.417385 2584 generic.go:356] "Generic (PLEG): container finished" podID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerID="bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77" exitCode=0 Jul 15 14:31:08.417585 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.417457 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" event={"ID":"d1671de3-2b58-4dc2-bfed-9d7467ba68aa","Type":"ContainerDied","Data":"bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77"} Jul 15 14:31:08.417866 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.417849 2584 scope.go:117] "RemoveContainer" containerID="bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77" Jul 15 14:31:08.418895 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.418866 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:08.418951 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.418938 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:08.418987 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:08.418973 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:09.424949 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:09.424909 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" event={"ID":"d1671de3-2b58-4dc2-bfed-9d7467ba68aa","Type":"ContainerStarted","Data":"c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176"} Jul 15 14:31:10.067659 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:10.067169 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:31:10.067659 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:10.067325 2584 patch_prober.go:28] interesting pod/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 15 14:31:10.067659 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:10.067384 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 15 14:31:11.067491 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:11.067450 2584 patch_prober.go:28] interesting pod/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 15 14:31:11.068086 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:11.068046 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 15 14:31:12.067737 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:12.067696 2584 patch_prober.go:28] interesting pod/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 15 14:31:12.068099 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:12.067761 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 15 14:31:13.067799 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:13.067761 2584 patch_prober.go:28] interesting pod/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 15 14:31:13.068192 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:13.067828 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 15 14:31:14.071637 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:14.071601 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:31:14.072078 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:14.072005 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:31:14.072641 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:14.072619 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:31:15.461543 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:15.461454 2584 generic.go:356] "Generic (PLEG): container finished" podID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerID="883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d" exitCode=0 Jul 15 14:31:15.461543 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:15.461527 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" event={"ID":"26acd0a8-5843-4f8f-8c38-8b5419d58482","Type":"ContainerDied","Data":"883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d"} Jul 15 14:31:15.462079 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:15.461948 2584 scope.go:117] "RemoveContainer" containerID="883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d" Jul 15 14:31:15.463006 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:15.462975 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:15.463078 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:15.463060 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:15.463127 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:15.463111 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 14:31:16.469645 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:16.469606 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" event={"ID":"26acd0a8-5843-4f8f-8c38-8b5419d58482","Type":"ContainerStarted","Data":"2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269"} Jul 15 14:31:16.798070 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:16.797977 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:16.799250 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:16.799227 2584 patch_prober.go:28] interesting pod/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 15 14:31:16.799375 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:16.799280 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 15 14:31:17.385662 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:17.385626 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:17.797514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:17.797421 2584 patch_prober.go:28] interesting pod/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 15 14:31:17.797514 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:17.797481 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 15 14:31:18.797579 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:18.797545 2584 patch_prober.go:28] interesting pod/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 15 14:31:18.797982 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:18.797596 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 15 14:31:19.797492 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:19.797454 2584 patch_prober.go:28] interesting pod/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 15 14:31:19.797691 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:19.797513 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 15 14:31:20.805083 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:20.805053 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:31:20.806062 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:31:20.806043 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:36:02.192862 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:36:02.192780 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:36:02.197967 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:36:02.197937 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:36:02.201361 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:36:02.201344 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:36:02.206376 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:36:02.206357 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p4t6d_039fd736-4e61-418b-9391-1d10f7401fc0/ovn-acl-logging/0.log" Jul 15 14:37:45.831500 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:45.831464 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7772-140622177901232"] Jul 15 14:37:45.835010 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:45.834993 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:45.837801 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:45.837776 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5rt4r\"" Jul 15 14:37:45.843375 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:45.843354 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177901232"] Jul 15 14:37:45.920882 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:45.920858 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6t2b\" (UniqueName: \"kubernetes.io/projected/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1-kube-api-access-f6t2b\") pod \"test-curl-7772-140622177901232\" (UID: \"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1\") " pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:46.022066 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:46.022041 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6t2b\" (UniqueName: \"kubernetes.io/projected/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1-kube-api-access-f6t2b\") pod \"test-curl-7772-140622177901232\" (UID: \"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1\") " pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:46.031752 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:46.031723 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6t2b\" (UniqueName: \"kubernetes.io/projected/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1-kube-api-access-f6t2b\") pod \"test-curl-7772-140622177901232\" (UID: \"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1\") " pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:46.145052 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:46.145021 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:46.280976 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:46.280952 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177901232"] Jul 15 14:37:46.282940 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:37:46.282914 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e430b3b_5479_4d80_81b1_8bfca6f8fdd1.slice/crio-3f820f06790eb428116d6e6b782be44b17202e9fa4333dab4cf4375172afafd6 WatchSource:0}: Error finding container 3f820f06790eb428116d6e6b782be44b17202e9fa4333dab4cf4375172afafd6: Status 404 returned error can't find the container with id 3f820f06790eb428116d6e6b782be44b17202e9fa4333dab4cf4375172afafd6 Jul 15 14:37:46.284252 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:46.284236 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 14:37:46.518045 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:46.517972 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177901232" event={"ID":"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1","Type":"ContainerStarted","Data":"3f820f06790eb428116d6e6b782be44b17202e9fa4333dab4cf4375172afafd6"} Jul 15 14:37:47.525007 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:47.524976 2584 generic.go:356] "Generic (PLEG): container finished" podID="6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" containerID="ea0053c367ab80857c332b3832cf22e143d69b894f05eeba445570dc4a5f5641" exitCode=0 Jul 15 14:37:47.525484 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:47.525071 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177901232" event={"ID":"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1","Type":"ContainerDied","Data":"ea0053c367ab80857c332b3832cf22e143d69b894f05eeba445570dc4a5f5641"} Jul 15 14:37:48.672175 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.672146 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:48.711596 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.711568 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7772-140622177901232_6e430b3b-5479-4d80-81b1-8bfca6f8fdd1/test-curl-7772-140622177901232/0.log" Jul 15 14:37:48.736902 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.736861 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7772-140622177901232"] Jul 15 14:37:48.739273 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.739241 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7772-140622177901232"] Jul 15 14:37:48.848779 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.848747 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6t2b\" (UniqueName: \"kubernetes.io/projected/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1-kube-api-access-f6t2b\") pod \"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1\" (UID: \"6e430b3b-5479-4d80-81b1-8bfca6f8fdd1\") " Jul 15 14:37:48.850874 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.850846 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1-kube-api-access-f6t2b" (OuterVolumeSpecName: "kube-api-access-f6t2b") pod "6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" (UID: "6e430b3b-5479-4d80-81b1-8bfca6f8fdd1"). InnerVolumeSpecName "kube-api-access-f6t2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:37:48.870161 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.870133 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7772-140622177902384"] Jul 15 14:37:48.870575 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.870561 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" containerName="test-curl-7772-140622177901232" Jul 15 14:37:48.870632 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.870576 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" containerName="test-curl-7772-140622177901232" Jul 15 14:37:48.870669 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.870642 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" containerName="test-curl-7772-140622177901232" Jul 15 14:37:48.873695 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.873679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:48.881055 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.881009 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177902384"] Jul 15 14:37:48.949621 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:48.949590 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6t2b\" (UniqueName: \"kubernetes.io/projected/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1-kube-api-access-f6t2b\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:37:49.050459 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.050416 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45tns\" (UniqueName: \"kubernetes.io/projected/6b69ad9a-0608-4c3a-8d70-84d274344e44-kube-api-access-45tns\") pod \"test-curl-7772-140622177902384\" (UID: \"6b69ad9a-0608-4c3a-8d70-84d274344e44\") " pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:49.151537 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.151439 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45tns\" (UniqueName: \"kubernetes.io/projected/6b69ad9a-0608-4c3a-8d70-84d274344e44-kube-api-access-45tns\") pod \"test-curl-7772-140622177902384\" (UID: \"6b69ad9a-0608-4c3a-8d70-84d274344e44\") " pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:49.160342 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.160292 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45tns\" (UniqueName: \"kubernetes.io/projected/6b69ad9a-0608-4c3a-8d70-84d274344e44-kube-api-access-45tns\") pod \"test-curl-7772-140622177902384\" (UID: \"6b69ad9a-0608-4c3a-8d70-84d274344e44\") " pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:49.184215 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.184176 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:49.319720 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.319684 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177902384"] Jul 15 14:37:49.323930 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:37:49.323903 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b69ad9a_0608_4c3a_8d70_84d274344e44.slice/crio-8e24acc12371af47ec88b68e9f5c34fb2df186e4f42103719ba4348bd30fc0fa WatchSource:0}: Error finding container 8e24acc12371af47ec88b68e9f5c34fb2df186e4f42103719ba4348bd30fc0fa: Status 404 returned error can't find the container with id 8e24acc12371af47ec88b68e9f5c34fb2df186e4f42103719ba4348bd30fc0fa Jul 15 14:37:49.536524 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.536417 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177902384" event={"ID":"6b69ad9a-0608-4c3a-8d70-84d274344e44","Type":"ContainerStarted","Data":"8e24acc12371af47ec88b68e9f5c34fb2df186e4f42103719ba4348bd30fc0fa"} Jul 15 14:37:49.537777 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.537753 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:49.537895 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:49.537781 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f820f06790eb428116d6e6b782be44b17202e9fa4333dab4cf4375172afafd6" Jul 15 14:37:49.550143 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:37:49.550116 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177901232\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" pod="opendatahub/test-curl-7772-140622177901232" Jul 15 14:37:50.007936 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:50.007906 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e430b3b-5479-4d80-81b1-8bfca6f8fdd1" path="/var/lib/kubelet/pods/6e430b3b-5479-4d80-81b1-8bfca6f8fdd1/volumes" Jul 15 14:37:50.544289 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:50.544258 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b69ad9a-0608-4c3a-8d70-84d274344e44" containerID="79662c5e06012fe3900966e7b81d96af44d8053177e470096ade55c24a9ed7d8" exitCode=0 Jul 15 14:37:50.544489 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:50.544328 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177902384" event={"ID":"6b69ad9a-0608-4c3a-8d70-84d274344e44","Type":"ContainerDied","Data":"79662c5e06012fe3900966e7b81d96af44d8053177e470096ade55c24a9ed7d8"} Jul 15 14:37:51.695860 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.695836 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:51.716900 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.716875 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7772-140622177902384_6b69ad9a-0608-4c3a-8d70-84d274344e44/test-curl-7772-140622177902384/0.log" Jul 15 14:37:51.743124 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.743090 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7772-140622177902384"] Jul 15 14:37:51.745026 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.745007 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7772-140622177902384"] Jul 15 14:37:51.775888 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.775869 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45tns\" (UniqueName: \"kubernetes.io/projected/6b69ad9a-0608-4c3a-8d70-84d274344e44-kube-api-access-45tns\") pod \"6b69ad9a-0608-4c3a-8d70-84d274344e44\" (UID: \"6b69ad9a-0608-4c3a-8d70-84d274344e44\") " Jul 15 14:37:51.777927 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.777901 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b69ad9a-0608-4c3a-8d70-84d274344e44-kube-api-access-45tns" (OuterVolumeSpecName: "kube-api-access-45tns") pod "6b69ad9a-0608-4c3a-8d70-84d274344e44" (UID: "6b69ad9a-0608-4c3a-8d70-84d274344e44"). InnerVolumeSpecName "kube-api-access-45tns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:37:51.877464 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:51.877429 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45tns\" (UniqueName: \"kubernetes.io/projected/6b69ad9a-0608-4c3a-8d70-84d274344e44-kube-api-access-45tns\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:37:52.014030 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:52.013998 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b69ad9a-0608-4c3a-8d70-84d274344e44" path="/var/lib/kubelet/pods/6b69ad9a-0608-4c3a-8d70-84d274344e44/volumes" Jul 15 14:37:52.555700 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:52.555670 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177902384" Jul 15 14:37:52.555902 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:52.555711 2584 scope.go:117] "RemoveContainer" containerID="79662c5e06012fe3900966e7b81d96af44d8053177e470096ade55c24a9ed7d8" Jul 15 14:37:58.206765 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.206732 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7772-140622177816416"] Jul 15 14:37:58.207141 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.207132 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b69ad9a-0608-4c3a-8d70-84d274344e44" containerName="test-curl-7772-140622177902384" Jul 15 14:37:58.207183 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.207143 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b69ad9a-0608-4c3a-8d70-84d274344e44" containerName="test-curl-7772-140622177902384" Jul 15 14:37:58.207227 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.207217 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b69ad9a-0608-4c3a-8d70-84d274344e44" containerName="test-curl-7772-140622177902384" Jul 15 14:37:58.211667 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.211650 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:37:58.214196 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.214174 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5rt4r\"" Jul 15 14:37:58.215797 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.215771 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"] Jul 15 14:37:58.233647 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.233619 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt27h\" (UniqueName: \"kubernetes.io/projected/ef4c17c6-e16a-48cd-8f26-3495d546914a-kube-api-access-dt27h\") pod \"test-curl-7772-140622177816416\" (UID: \"ef4c17c6-e16a-48cd-8f26-3495d546914a\") " pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:37:58.334877 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.334837 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt27h\" (UniqueName: \"kubernetes.io/projected/ef4c17c6-e16a-48cd-8f26-3495d546914a-kube-api-access-dt27h\") pod \"test-curl-7772-140622177816416\" (UID: \"ef4c17c6-e16a-48cd-8f26-3495d546914a\") " pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:37:58.343851 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.343824 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt27h\" (UniqueName: \"kubernetes.io/projected/ef4c17c6-e16a-48cd-8f26-3495d546914a-kube-api-access-dt27h\") pod \"test-curl-7772-140622177816416\" (UID: \"ef4c17c6-e16a-48cd-8f26-3495d546914a\") " pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:37:58.522366 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.522265 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:37:58.664383 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:58.664337 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"] Jul 15 14:37:58.665658 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:37:58.665631 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef4c17c6_e16a_48cd_8f26_3495d546914a.slice/crio-e68ccdb8f073fc6f13d13e44cdb0d12f0879708e09bf307965e4d230bab71325 WatchSource:0}: Error finding container e68ccdb8f073fc6f13d13e44cdb0d12f0879708e09bf307965e4d230bab71325: Status 404 returned error can't find the container with id e68ccdb8f073fc6f13d13e44cdb0d12f0879708e09bf307965e4d230bab71325 Jul 15 14:37:59.597182 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:59.597148 2584 generic.go:356] "Generic (PLEG): container finished" podID="ef4c17c6-e16a-48cd-8f26-3495d546914a" containerID="5b1598b8c9afcf8f5d5e897392dcf8a22b4d2913412050ad9d259ec306c6594e" exitCode=0 Jul 15 14:37:59.597592 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:59.597214 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177816416" event={"ID":"ef4c17c6-e16a-48cd-8f26-3495d546914a","Type":"ContainerDied","Data":"5b1598b8c9afcf8f5d5e897392dcf8a22b4d2913412050ad9d259ec306c6594e"} Jul 15 14:37:59.597592 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:37:59.597247 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7772-140622177816416" event={"ID":"ef4c17c6-e16a-48cd-8f26-3495d546914a","Type":"ContainerStarted","Data":"e68ccdb8f073fc6f13d13e44cdb0d12f0879708e09bf307965e4d230bab71325"} Jul 15 14:38:00.748101 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.748075 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:38:00.770096 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.770060 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7772-140622177816416_ef4c17c6-e16a-48cd-8f26-3495d546914a/test-curl-7772-140622177816416/0.log" Jul 15 14:38:00.798137 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.798104 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"] Jul 15 14:38:00.804209 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.804186 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7772-140622177816416"] Jul 15 14:38:00.859060 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.858992 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt27h\" (UniqueName: \"kubernetes.io/projected/ef4c17c6-e16a-48cd-8f26-3495d546914a-kube-api-access-dt27h\") pod \"ef4c17c6-e16a-48cd-8f26-3495d546914a\" (UID: \"ef4c17c6-e16a-48cd-8f26-3495d546914a\") " Jul 15 14:38:00.861011 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.860988 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef4c17c6-e16a-48cd-8f26-3495d546914a-kube-api-access-dt27h" (OuterVolumeSpecName: "kube-api-access-dt27h") pod "ef4c17c6-e16a-48cd-8f26-3495d546914a" (UID: "ef4c17c6-e16a-48cd-8f26-3495d546914a"). InnerVolumeSpecName "kube-api-access-dt27h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:38:00.959926 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:00.959895 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt27h\" (UniqueName: \"kubernetes.io/projected/ef4c17c6-e16a-48cd-8f26-3495d546914a-kube-api-access-dt27h\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:01.608755 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:01.608728 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e68ccdb8f073fc6f13d13e44cdb0d12f0879708e09bf307965e4d230bab71325" Jul 15 14:38:01.608755 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:01.608746 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:38:01.618894 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:38:01.618865 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177816416\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ef4c17c6-e16a-48cd-8f26-3495d546914a" pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:38:02.007585 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:38:02.007496 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7772-140622177816416\" is forbidden: User \"system:node:ip-10-0-133-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-21.ec2.internal' and this object" podUID="ef4c17c6-e16a-48cd-8f26-3495d546914a" pod="opendatahub/test-curl-7772-140622177816416" Jul 15 14:38:02.008876 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:02.008855 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef4c17c6-e16a-48cd-8f26-3495d546914a" path="/var/lib/kubelet/pods/ef4c17c6-e16a-48cd-8f26-3495d546914a/volumes" Jul 15 14:38:30.191173 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:30.191142 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px"] Jul 15 14:38:30.191785 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:30.191417 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" containerID="cri-o://c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176" gracePeriod=30 Jul 15 14:38:35.454980 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.454923 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:38:35.502773 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.502737 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47ttg\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-kube-api-access-47ttg\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.502932 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.502821 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-podinfo\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.502932 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.502860 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-certs\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.502932 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.502889 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-socket\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.502932 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.502919 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istiod-ca-cert\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.503140 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.502978 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-credential-socket\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.503140 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503040 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-data\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.503140 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503094 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-token\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.503311 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503138 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-envoy\") pod \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\" (UID: \"d1671de3-2b58-4dc2-bfed-9d7467ba68aa\") " Jul 15 14:38:35.503311 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503186 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:38:35.503612 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503513 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.503612 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503546 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:38:35.503746 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503719 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:38:35.503805 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503772 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:38:35.503976 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.503950 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:38:35.505529 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.505503 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-kube-api-access-47ttg" (OuterVolumeSpecName: "kube-api-access-47ttg") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "kube-api-access-47ttg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:38:35.505750 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.505721 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:38:35.505750 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.505737 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:38:35.505921 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.505902 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1671de3-2b58-4dc2-bfed-9d7467ba68aa" (UID: "d1671de3-2b58-4dc2-bfed-9d7467ba68aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:38:35.604485 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604455 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47ttg\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-kube-api-access-47ttg\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604485 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604483 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604485 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604494 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604700 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604502 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604700 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604512 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604700 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604520 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604700 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604527 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.604700 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.604536 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1671de3-2b58-4dc2-bfed-9d7467ba68aa-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:38:35.786137 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.786054 2584 generic.go:356] "Generic (PLEG): container finished" podID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerID="c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176" exitCode=0 Jul 15 14:38:35.786137 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.786102 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" Jul 15 14:38:35.786384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.786100 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" event={"ID":"d1671de3-2b58-4dc2-bfed-9d7467ba68aa","Type":"ContainerDied","Data":"c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176"} Jul 15 14:38:35.786384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.786248 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px" event={"ID":"d1671de3-2b58-4dc2-bfed-9d7467ba68aa","Type":"ContainerDied","Data":"fcf26727e23c82d2e796f0dbd3f02fa9c9b7b327d344a35d678eca1330d6c60c"} Jul 15 14:38:35.786384 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.786288 2584 scope.go:117] "RemoveContainer" containerID="c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176" Jul 15 14:38:35.796032 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.796015 2584 scope.go:117] "RemoveContainer" containerID="bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77" Jul 15 14:38:35.804370 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.804352 2584 scope.go:117] "RemoveContainer" containerID="c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176" Jul 15 14:38:35.804613 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:38:35.804596 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176\": container with ID starting with c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176 not found: ID does not exist" containerID="c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176" Jul 15 14:38:35.804665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.804621 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176"} err="failed to get container status \"c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176\": rpc error: code = NotFound desc = could not find container \"c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176\": container with ID starting with c5071db3e444a0bee585f34cce3ee1486933782b06a9b32d0d53d10d7799a176 not found: ID does not exist" Jul 15 14:38:35.804665 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.804637 2584 scope.go:117] "RemoveContainer" containerID="bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77" Jul 15 14:38:35.804857 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:38:35.804839 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77\": container with ID starting with bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77 not found: ID does not exist" containerID="bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77" Jul 15 14:38:35.804904 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.804864 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77"} err="failed to get container status \"bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77\": rpc error: code = NotFound desc = could not find container \"bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77\": container with ID starting with bb5a3fcdba4dec79c8a6bf5b6979c7935a80f93b382f3df41a97464292cb3b77 not found: ID does not exist" Jul 15 14:38:35.809624 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.809603 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px"] Jul 15 14:38:35.813891 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:35.813871 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0b4a59-openshift-default-75b847bd7-4j8px"] Jul 15 14:38:36.009080 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:36.009046 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" path="/var/lib/kubelet/pods/d1671de3-2b58-4dc2-bfed-9d7467ba68aa/volumes" Jul 15 14:38:56.433596 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:56.433513 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758"] Jul 15 14:38:56.434026 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:56.433792 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" containerID="cri-o://2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269" gracePeriod=30 Jul 15 14:38:57.203860 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:38:57.203828 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57864c88fb-2rcr8_5c833a98-9507-4386-b9d5-20041ec59999/authorino/0.log" Jul 15 14:39:01.694321 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.694281 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:39:01.831034 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.830949 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-data\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831034 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831006 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-certs\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831034 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831027 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-podinfo\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831329 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831064 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-token\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831329 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831100 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-credential-socket\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831329 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831127 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-socket\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831329 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831160 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-envoy\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831329 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831187 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pcvp\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-kube-api-access-8pcvp\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831329 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831218 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26acd0a8-5843-4f8f-8c38-8b5419d58482-istiod-ca-cert\") pod \"26acd0a8-5843-4f8f-8c38-8b5419d58482\" (UID: \"26acd0a8-5843-4f8f-8c38-8b5419d58482\") " Jul 15 14:39:01.831663 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831329 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:39:01.831663 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831568 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:39:01.831663 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831589 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:39:01.831663 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831645 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-certs\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.831930 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831904 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26acd0a8-5843-4f8f-8c38-8b5419d58482-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 14:39:01.832015 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.831976 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-data" (OuterVolumeSpecName: "istio-data") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:39:01.833537 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.833513 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 14:39:01.833636 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.833513 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 14:39:01.833755 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.833735 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-token" (OuterVolumeSpecName: "istio-token") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:39:01.833755 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.833745 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-kube-api-access-8pcvp" (OuterVolumeSpecName: "kube-api-access-8pcvp") pod "26acd0a8-5843-4f8f-8c38-8b5419d58482" (UID: "26acd0a8-5843-4f8f-8c38-8b5419d58482"). InnerVolumeSpecName "kube-api-access-8pcvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 14:39:01.928976 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.928944 2584 generic.go:356] "Generic (PLEG): container finished" podID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerID="2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269" exitCode=0 Jul 15 14:39:01.929135 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.928999 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" Jul 15 14:39:01.929135 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.929037 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" event={"ID":"26acd0a8-5843-4f8f-8c38-8b5419d58482","Type":"ContainerDied","Data":"2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269"} Jul 15 14:39:01.929135 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.929109 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758" event={"ID":"26acd0a8-5843-4f8f-8c38-8b5419d58482","Type":"ContainerDied","Data":"25c96b9fd867dabda34b25a04170711c984d99ddf0b51221fae459223d3bf24c"} Jul 15 14:39:01.929243 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.929140 2584 scope.go:117] "RemoveContainer" containerID="2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269" Jul 15 14:39:01.932458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932435 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-data\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932458 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932459 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-podinfo\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932473 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-token\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932487 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-credential-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932501 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-workload-socket\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932514 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26acd0a8-5843-4f8f-8c38-8b5419d58482-istio-envoy\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932529 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pcvp\" (UniqueName: \"kubernetes.io/projected/26acd0a8-5843-4f8f-8c38-8b5419d58482-kube-api-access-8pcvp\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.932652 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.932544 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26acd0a8-5843-4f8f-8c38-8b5419d58482-istiod-ca-cert\") on node \"ip-10-0-133-21.ec2.internal\" DevicePath \"\"" Jul 15 14:39:01.939325 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.939284 2584 scope.go:117] "RemoveContainer" containerID="883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d" Jul 15 14:39:01.947250 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.947233 2584 scope.go:117] "RemoveContainer" containerID="2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269" Jul 15 14:39:01.947501 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:39:01.947483 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269\": container with ID starting with 2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269 not found: ID does not exist" containerID="2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269" Jul 15 14:39:01.947596 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.947512 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269"} err="failed to get container status \"2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269\": rpc error: code = NotFound desc = could not find container \"2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269\": container with ID starting with 2922f151ed755c5ad809183447590d6a958edf3365c4071152cb98d91b622269 not found: ID does not exist" Jul 15 14:39:01.947596 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.947538 2584 scope.go:117] "RemoveContainer" containerID="883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d" Jul 15 14:39:01.947765 ip-10-0-133-21 kubenswrapper[2584]: E0715 14:39:01.947750 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d\": container with ID starting with 883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d not found: ID does not exist" containerID="883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d" Jul 15 14:39:01.947818 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.947772 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d"} err="failed to get container status \"883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d\": rpc error: code = NotFound desc = could not find container \"883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d\": container with ID starting with 883e8836bba8986f14f65c02cec853a55155580bbb3494e387159e738969506d not found: ID does not exist" Jul 15 14:39:01.955025 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.955004 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758"] Jul 15 14:39:01.961539 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:01.961518 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0a4dfd-openshift-default-67946db58c-v9758"] Jul 15 14:39:02.007348 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:02.007323 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" path="/var/lib/kubelet/pods/26acd0a8-5843-4f8f-8c38-8b5419d58482/volumes" Jul 15 14:39:02.677056 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:02.676982 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-78pn9_aa4ca8eb-ad72-497a-9d07-1570602a84ab/manager/0.log" Jul 15 14:39:02.809114 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:02.809076 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rw87m_64f00680-3358-41e7-80f2-efcd392d726e/manager/0.log" Jul 15 14:39:02.942273 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:02.942194 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-688d755c47-tm2lw_22a2b338-2612-49d6-8c1b-3d73d18d6696/manager/0.log" Jul 15 14:39:03.063645 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:03.063619 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fgj4p_1f37dd4f-e424-4acd-a5b6-3eafe51a31e3/server/0.log" Jul 15 14:39:03.217764 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:03.217674 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bpv62_11a36d1b-a345-431f-a247-25f53942bf84/manager/0.log" Jul 15 14:39:03.504345 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:03.504233 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-5sxk4_79cc6a28-8d13-4f87-931e-e108f64edffc/copy-manifests/0.log" Jul 15 14:39:03.519695 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:03.519671 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-5sxk4_79cc6a28-8d13-4f87-931e-e108f64edffc/manager/0.log" Jul 15 14:39:04.113371 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:04.113335 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-d75896c4d-r7dbl_19947c84-8671-40d9-8124-26148f6491fb/maas-api/0.log" Jul 15 14:39:04.238631 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:04.238603 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735415-d7f7z_69e315b9-9a9f-42b5-9d34-3747e3f56076/cleanup/0.log" Jul 15 14:39:06.092046 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:06.092008 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ndsfz_3f6bf743-bcef-4eeb-b3f0-516365bc09d7/postgres/0.log" Jul 15 14:39:07.580210 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:07.580180 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57864c88fb-2rcr8_5c833a98-9507-4386-b9d5-20041ec59999/authorino/0.log" Jul 15 14:39:07.729335 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:07.729311 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hq6vk_5da529bc-a05d-4c68-81f7-37955bd42359/manager/0.log" Jul 15 14:39:07.859191 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:07.859167 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fzdk2_c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7/manager/0.log" Jul 15 14:39:08.134888 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:08.134800 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pl69n_03642870-f19d-4884-9373-d751cd253ea5/registry-server/0.log" Jul 15 14:39:08.269036 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:08.269013 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zk9hj_10016cb2-44b9-460d-9281-bd4fe2b596b8/manager/0.log" Jul 15 14:39:09.220074 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:09.220043 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-r7g6z_2e2307eb-e699-49e0-bb35-821e6d2bfa9c/kube-auth-proxy/0.log" Jul 15 14:39:09.465147 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:09.465122 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/1.log" Jul 15 14:39:09.735927 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:09.735898 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s6fqz_501a2552-af60-4325-ac18-7615b83bcadb/payload-processing/0.log" Jul 15 14:39:09.864115 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:09.864079 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66c5645f8-m26q2_864878a2-8862-420c-886d-0d7b2cf462c9/router/0.log" Jul 15 14:39:10.267087 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:10.267066 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bt8kc_027b2e2b-3c0d-45d5-a956-d2167a262861/main/0.log" Jul 15 14:39:10.523453 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:10.523382 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jflvh_d02b4d95-a1ea-4183-8b7b-728a39a4de14/main/0.log" Jul 15 14:39:19.998773 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:19.998744 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t7tnx_cf75f52c-b77a-4a9a-8b02-7e823c68380a/global-pull-secret-syncer/0.log" Jul 15 14:39:20.163634 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:20.163610 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-858gf_5707ea40-64ff-4bf3-b355-32b3d29f80f8/konnectivity-agent/0.log" Jul 15 14:39:20.247990 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:20.247964 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-21.ec2.internal_72a36e4b47671bef90a84d11574ce86d/haproxy/0.log" Jul 15 14:39:25.551495 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:25.551440 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57864c88fb-2rcr8_5c833a98-9507-4386-b9d5-20041ec59999/authorino/0.log" Jul 15 14:39:25.608253 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:25.608228 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hq6vk_5da529bc-a05d-4c68-81f7-37955bd42359/manager/0.log" Jul 15 14:39:25.696818 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:25.696795 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fzdk2_c0cc3ef1-49e1-45b3-8194-4f6a1d0655f7/manager/0.log" Jul 15 14:39:25.815627 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:25.815562 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pl69n_03642870-f19d-4884-9373-d751cd253ea5/registry-server/0.log" Jul 15 14:39:25.959915 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:25.959886 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zk9hj_10016cb2-44b9-460d-9281-bd4fe2b596b8/manager/0.log" Jul 15 14:39:28.178900 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:28.178872 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-6tmpw_516d2f7f-144e-4862-9e00-fc044804030a/cluster-monitoring-operator/0.log" Jul 15 14:39:28.759455 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:28.759425 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q9php_02f12689-395c-444a-996e-e539ca2ab968/node-exporter/0.log" Jul 15 14:39:28.781161 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:28.781141 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q9php_02f12689-395c-444a-996e-e539ca2ab968/kube-rbac-proxy/0.log" Jul 15 14:39:28.818428 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:28.818400 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q9php_02f12689-395c-444a-996e-e539ca2ab968/init-textfile/0.log" Jul 15 14:39:31.133206 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.133177 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/2.log" Jul 15 14:39:31.136988 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.136972 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pcv4_9f828c0e-e03b-4683-8405-028251958c94/console-operator/3.log" Jul 15 14:39:31.377495 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377465 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl"] Jul 15 14:39:31.377870 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377858 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377872 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377886 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377891 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377898 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377903 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377915 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef4c17c6-e16a-48cd-8f26-3495d546914a" containerName="test-curl-7772-140622177816416" Jul 15 14:39:31.377923 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377921 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef4c17c6-e16a-48cd-8f26-3495d546914a" containerName="test-curl-7772-140622177816416" Jul 15 14:39:31.378138 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377982 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef4c17c6-e16a-48cd-8f26-3495d546914a" containerName="test-curl-7772-140622177816416" Jul 15 14:39:31.378138 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377991 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="26acd0a8-5843-4f8f-8c38-8b5419d58482" containerName="istio-proxy" Jul 15 14:39:31.378138 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.377996 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" Jul 15 14:39:31.378138 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.378003 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1671de3-2b58-4dc2-bfed-9d7467ba68aa" containerName="istio-proxy" Jul 15 14:39:31.381225 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.381206 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.384749 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.384695 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tzfdj\"/\"default-dockercfg-v7lr9\"" Jul 15 14:39:31.384835 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.384758 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tzfdj\"/\"openshift-service-ca.crt\"" Jul 15 14:39:31.385776 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.385757 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tzfdj\"/\"kube-root-ca.crt\"" Jul 15 14:39:31.391572 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.391549 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl"] Jul 15 14:39:31.489021 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.488990 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtbs6\" (UniqueName: \"kubernetes.io/projected/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-kube-api-access-xtbs6\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.489192 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.489032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-podres\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.489192 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.489058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-proc\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.489192 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.489148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-lib-modules\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.489331 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.489220 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-sys\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590199 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-sys\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590211 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtbs6\" (UniqueName: \"kubernetes.io/projected/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-kube-api-access-xtbs6\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590252 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-podres\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-proc\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590288 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-sys\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590382 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-lib-modules\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590638 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590428 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-proc\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590638 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590426 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-podres\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.590638 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.590573 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-lib-modules\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.601984 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.601958 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtbs6\" (UniqueName: \"kubernetes.io/projected/7000318c-f6b4-4c2d-9ca6-8ec2b468805b-kube-api-access-xtbs6\") pod \"perf-node-gather-daemonset-blzwl\" (UID: \"7000318c-f6b4-4c2d-9ca6-8ec2b468805b\") " pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.687694 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.687610 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5876bcc5cb-pv6sr_e149cbc9-ce22-4e6b-a4de-78f1fb9c6ad0/console/0.log" Jul 15 14:39:31.691508 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.691491 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:31.826747 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:31.826701 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl"] Jul 15 14:39:31.828586 ip-10-0-133-21 kubenswrapper[2584]: W0715 14:39:31.828551 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7000318c_f6b4_4c2d_9ca6_8ec2b468805b.slice/crio-8f29655258fa96097b7bf7a866d4d5362201175bf40047dee93d243506dce7a8 WatchSource:0}: Error finding container 8f29655258fa96097b7bf7a866d4d5362201175bf40047dee93d243506dce7a8: Status 404 returned error can't find the container with id 8f29655258fa96097b7bf7a866d4d5362201175bf40047dee93d243506dce7a8 Jul 15 14:39:32.079255 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:32.079215 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" event={"ID":"7000318c-f6b4-4c2d-9ca6-8ec2b468805b","Type":"ContainerStarted","Data":"807e969b50cb23255398d7db04cdc896be219d8ea65fefe39efb62347627255e"} Jul 15 14:39:32.079463 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:32.079265 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" event={"ID":"7000318c-f6b4-4c2d-9ca6-8ec2b468805b","Type":"ContainerStarted","Data":"8f29655258fa96097b7bf7a866d4d5362201175bf40047dee93d243506dce7a8"} Jul 15 14:39:32.079463 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:32.079321 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:32.101014 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:32.100959 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" podStartSLOduration=1.100946652 podStartE2EDuration="1.100946652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 14:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:39:32.099815406 +0000 UTC m=+2910.537424200" watchObservedRunningTime="2026-07-15 14:39:32.100946652 +0000 UTC m=+2910.538555443" Jul 15 14:39:32.379140 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:32.379111 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-h2twb_d3f3476a-97d2-415c-ab0a-ff9fd002ff50/volume-data-source-validator/0.log" Jul 15 14:39:33.473699 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:33.473672 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dsffv_9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5/dns/0.log" Jul 15 14:39:33.495107 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:33.495080 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dsffv_9d5b5684-9f66-4e5a-ac0c-94b9f1f4a8b5/kube-rbac-proxy/0.log" Jul 15 14:39:33.625405 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:33.625377 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9whzn_9618dd0e-33ad-4458-88d6-7442904afcf4/dns-node-resolver/0.log" Jul 15 14:39:34.274244 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:34.274210 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74ddfb6f4d-j8lvr_7177928d-8085-4aea-b14a-bb17c4dca4d9/registry/0.log" Jul 15 14:39:34.305644 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:34.305616 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fl65z_4796b9f6-2024-4e6c-a008-8f3060ea1b4e/node-ca/0.log" Jul 15 14:39:35.519666 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:35.519636 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-r7g6z_2e2307eb-e699-49e0-bb35-821e6d2bfa9c/kube-auth-proxy/0.log" Jul 15 14:39:35.586460 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:35.586415 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/0.log" Jul 15 14:39:35.615506 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:35.615478 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/1.log" Jul 15 14:39:35.683836 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:35.683805 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s6fqz_501a2552-af60-4325-ac18-7615b83bcadb/payload-processing/0.log" Jul 15 14:39:35.706548 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:35.706512 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66c5645f8-m26q2_864878a2-8862-420c-886d-0d7b2cf462c9/router/0.log" Jul 15 14:39:36.573780 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:36.573753 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zmd4q_43e414e3-fc9e-40ee-912a-6597bd9d2faf/serve-healthcheck-canary/0.log" Jul 15 14:39:37.070894 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:37.070862 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7ddh9_47accf41-e456-4b19-9cb2-edb9c0369ac2/insights-operator/0.log" Jul 15 14:39:37.071697 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:37.071679 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7ddh9_47accf41-e456-4b19-9cb2-edb9c0369ac2/insights-operator/1.log" Jul 15 14:39:37.196813 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:37.196786 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-swh9f_facfeec1-06ad-463c-8d29-55dd1333df11/kube-rbac-proxy/0.log" Jul 15 14:39:37.223197 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:37.223174 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-swh9f_facfeec1-06ad-463c-8d29-55dd1333df11/exporter/0.log" Jul 15 14:39:37.247992 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:37.247972 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-swh9f_facfeec1-06ad-463c-8d29-55dd1333df11/extractor/0.log" Jul 15 14:39:38.093836 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:38.093811 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tzfdj/perf-node-gather-daemonset-blzwl" Jul 15 14:39:39.569318 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:39.569273 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-78pn9_aa4ca8eb-ad72-497a-9d07-1570602a84ab/manager/0.log" Jul 15 14:39:39.700942 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:39.700917 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rw87m_64f00680-3358-41e7-80f2-efcd392d726e/manager/0.log" Jul 15 14:39:39.804347 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:39.804319 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-688d755c47-tm2lw_22a2b338-2612-49d6-8c1b-3d73d18d6696/manager/0.log" Jul 15 14:39:39.828556 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:39.828482 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fgj4p_1f37dd4f-e424-4acd-a5b6-3eafe51a31e3/server/0.log" Jul 15 14:39:39.901589 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:39.901563 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bpv62_11a36d1b-a345-431f-a247-25f53942bf84/manager/0.log" Jul 15 14:39:40.038471 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:40.038442 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-5sxk4_79cc6a28-8d13-4f87-931e-e108f64edffc/manager/0.log" Jul 15 14:39:40.058482 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:40.058457 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58577d4d66-5sxk4_79cc6a28-8d13-4f87-931e-e108f64edffc/copy-manifests/0.log" Jul 15 14:39:41.553836 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:41.553806 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-r7g6z_2e2307eb-e699-49e0-bb35-821e6d2bfa9c/kube-auth-proxy/0.log" Jul 15 14:39:41.606101 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:41.606078 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/0.log" Jul 15 14:39:41.634580 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:41.634559 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/1.log" Jul 15 14:39:41.678201 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:41.678174 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s6fqz_501a2552-af60-4325-ac18-7615b83bcadb/payload-processing/0.log" Jul 15 14:39:41.692607 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:41.692577 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66c5645f8-m26q2_864878a2-8862-420c-886d-0d7b2cf462c9/router/0.log" Jul 15 14:39:41.696221 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:41.696186 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wq8q7_2ae1a253-077e-4c47-8f81-aed0b24184b8/openshift-lws-operator/0.log" Jul 15 14:39:43.086313 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:43.086279 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b5998d775-r7g6z_2e2307eb-e699-49e0-bb35-821e6d2bfa9c/kube-auth-proxy/0.log" Jul 15 14:39:43.129363 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:43.129338 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/0.log" Jul 15 14:39:43.158216 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:43.158194 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jgfff_c637485c-0a7e-45da-9344-046ab1f44771/istio-proxy/1.log" Jul 15 14:39:43.204836 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:43.204774 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s6fqz_501a2552-af60-4325-ac18-7615b83bcadb/payload-processing/0.log" Jul 15 14:39:43.220690 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:43.220662 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66c5645f8-m26q2_864878a2-8862-420c-886d-0d7b2cf462c9/router/0.log" Jul 15 14:39:47.370131 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:47.370092 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-85mvn_54a8e011-19a1-4f5e-a00d-e7f937aab4d2/kube-storage-version-migrator-operator/1.log" Jul 15 14:39:47.370821 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:47.370802 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-85mvn_54a8e011-19a1-4f5e-a00d-e7f937aab4d2/kube-storage-version-migrator-operator/0.log" Jul 15 14:39:48.502683 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.502654 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/kube-multus-additional-cni-plugins/0.log" Jul 15 14:39:48.526381 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.526359 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/egress-router-binary-copy/0.log" Jul 15 14:39:48.550037 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.550013 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/cni-plugins/0.log" Jul 15 14:39:48.573180 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.573153 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/bond-cni-plugin/0.log" Jul 15 14:39:48.595839 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.595815 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/routeoverride-cni/0.log" Jul 15 14:39:48.619878 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.619849 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/whereabouts-cni-bincopy/0.log" Jul 15 14:39:48.649685 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:48.649662 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26sc2_01aac545-69a2-4eda-97dc-e2a200140a51/whereabouts-cni/0.log" Jul 15 14:39:49.083027 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:49.083001 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k4k64_f8be98fb-fca8-4048-b97e-b042af16da82/kube-multus/0.log" Jul 15 14:39:49.206510 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:49.206479 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ncnc9_b731e8a7-a469-4d64-9ac5-14ced255483d/network-metrics-daemon/0.log" Jul 15 14:39:49.227913 ip-10-0-133-21 kubenswrapper[2584]: I0715 14:39:49.227887 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ncnc9_b731e8a7-a469-4d64-9ac5-14ced255483d/kube-rbac-proxy/0.log"