Jul 20 17:56:10.753482 ip-10-0-133-135 systemd[1]: Starting Kubernetes Kubelet... Jul 20 17:56:11.167256 ip-10-0-133-135 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 17:56:11.167256 ip-10-0-133-135 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 17:56:11.167256 ip-10-0-133-135 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 17:56:11.167256 ip-10-0-133-135 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 17:56:11.167256 ip-10-0-133-135 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 17:56:11.169387 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.169271 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173729 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173753 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173759 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173763 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173768 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173773 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173783 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173788 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173793 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173797 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173802 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173806 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173810 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173817 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173823 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173833 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173838 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173842 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 17:56:11.173931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173850 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173861 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173866 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173871 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173876 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173881 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173885 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.173926 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174463 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174475 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174481 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174486 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174491 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174497 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174500 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174504 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174507 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 17:56:11.174500 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174510 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174513 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174516 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174518 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174521 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174524 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174526 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174529 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174532 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174535 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174537 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174540 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174543 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174545 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174549 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174552 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174555 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174558 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174560 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174563 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 17:56:11.174908 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174565 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174568 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174571 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174574 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174579 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174581 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174586 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174591 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174594 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174597 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174600 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174604 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174608 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174611 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174615 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174618 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174621 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174624 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174627 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 17:56:11.175431 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174630 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174633 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174637 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174640 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174643 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174645 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174648 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174651 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174653 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174656 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174658 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174661 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174664 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174667 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174669 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174672 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174675 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174677 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174680 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174682 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 17:56:11.175882 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174685 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174688 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174691 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174694 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.174696 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176003 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176012 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176016 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176019 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176021 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176024 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176027 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176029 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176032 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176034 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176038 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176045 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176049 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176053 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176056 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 17:56:11.176399 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176058 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176061 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176064 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176067 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176069 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176072 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176074 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176076 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176079 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176081 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176084 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176087 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176089 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176091 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176095 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176098 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176101 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176103 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176105 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176108 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 17:56:11.176946 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176111 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176113 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176116 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176118 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176121 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176123 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176125 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176130 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176133 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176136 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176139 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176141 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176144 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176146 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176148 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176151 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176154 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176156 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176159 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 17:56:11.177486 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176161 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176163 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176166 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176168 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176171 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176173 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176176 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176197 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176200 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176203 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176205 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176208 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176210 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176213 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176216 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176219 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176221 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176224 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176226 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176229 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 17:56:11.177934 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176231 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176234 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176236 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176239 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176241 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176244 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176246 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176250 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176253 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176256 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176259 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176263 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176266 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176270 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176273 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176276 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176278 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176281 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176284 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 17:56:11.178435 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176287 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176289 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176292 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176295 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176297 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176300 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176706 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176712 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176715 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176718 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176721 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176724 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176727 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176729 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176732 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176735 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176737 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176740 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176743 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176746 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 17:56:11.178890 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176750 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176753 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176759 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176762 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176766 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176769 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176771 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176774 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176777 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176779 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176782 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176785 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176788 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176790 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176793 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176795 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176798 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176800 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176803 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176806 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 17:56:11.179392 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176808 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176811 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176813 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176816 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176818 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176821 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176823 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176826 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176828 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176831 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176833 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176836 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176839 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176841 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176844 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176847 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176850 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176853 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176856 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176858 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 17:56:11.179860 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176861 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176863 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176866 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176869 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176872 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176874 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176877 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176879 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176882 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176885 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176887 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176890 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176893 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176895 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176898 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176900 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176902 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176906 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176909 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 17:56:11.180351 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176912 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176916 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176919 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176921 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176924 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176926 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176929 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176932 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176935 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176941 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176944 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176946 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176949 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176951 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176954 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176957 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176964 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176968 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176970 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 17:56:11.180792 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176973 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176976 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176978 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176981 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176984 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176986 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.176989 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177059 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177067 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177078 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177082 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177086 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177089 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177093 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177097 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177100 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177103 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177107 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177110 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177113 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177116 2567 flags.go:64] FLAG: --cgroup-root="" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177119 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177122 2567 flags.go:64] FLAG: --client-ca-file="" Jul 20 17:56:11.181241 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177127 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177134 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177150 2567 flags.go:64] FLAG: --cluster-domain="" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177155 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177158 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177161 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177166 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177170 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177173 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177188 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177191 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177194 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177198 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177201 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177204 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177207 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177211 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177214 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177217 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177219 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177222 2567 flags.go:64] FLAG: --enable-server="true" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177226 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177231 2567 flags.go:64] FLAG: --event-burst="100" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177234 2567 flags.go:64] FLAG: --event-qps="50" Jul 20 17:56:11.181784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177237 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177240 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177243 2567 flags.go:64] FLAG: --eviction-hard="" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177247 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177250 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177253 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177256 2567 flags.go:64] FLAG: --eviction-soft="" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177259 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177262 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177270 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177273 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177276 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177279 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177282 2567 flags.go:64] FLAG: --feature-gates="" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177286 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177289 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177292 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177296 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177299 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177302 2567 flags.go:64] FLAG: --help="false" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177305 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177308 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177311 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 17:56:11.182361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177314 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177317 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177321 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177324 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177326 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177329 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177332 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177335 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177338 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177341 2567 flags.go:64] FLAG: --kube-reserved="" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177344 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177346 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177350 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177353 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177356 2567 flags.go:64] FLAG: --lock-file="" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177359 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177362 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177365 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177371 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177375 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177378 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177381 2567 flags.go:64] FLAG: --logging-format="text" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177384 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177389 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 17:56:11.182977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177391 2567 flags.go:64] FLAG: --manifest-url="" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177394 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177398 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177402 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177406 2567 flags.go:64] FLAG: --max-pods="110" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177409 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177412 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177415 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177418 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177421 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177424 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177426 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177434 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177438 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177441 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177444 2567 flags.go:64] FLAG: --pod-cidr="" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177447 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177452 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177455 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177458 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177461 2567 flags.go:64] FLAG: --port="10250" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177464 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177467 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0686a28ab28700ed8" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177470 2567 flags.go:64] FLAG: --qos-reserved="" Jul 20 17:56:11.183572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177473 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177476 2567 flags.go:64] FLAG: --register-node="true" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177480 2567 flags.go:64] FLAG: --register-with-taints="" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177483 2567 flags.go:64] FLAG: --registry-burst="10" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177487 2567 flags.go:64] FLAG: --registry-qps="5" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177490 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177493 2567 flags.go:64] FLAG: --reserved-memory="" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177497 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177501 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177504 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177507 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177510 2567 flags.go:64] FLAG: --runonce="false" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177513 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177516 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177519 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177522 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177525 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177528 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177531 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177534 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177537 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177540 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177543 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177545 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177548 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 17:56:11.184161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177551 2567 flags.go:64] FLAG: --system-cgroups="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177554 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177559 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177562 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177565 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177569 2567 flags.go:64] FLAG: --tls-min-version="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177572 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177574 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177577 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177581 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177584 2567 flags.go:64] FLAG: --v="2" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177588 2567 flags.go:64] FLAG: --version="false" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177593 2567 flags.go:64] FLAG: --vmodule="" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177596 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.177600 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177726 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177730 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177733 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177736 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177739 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177742 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177744 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177747 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 17:56:11.184778 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177749 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177752 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177754 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177757 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177759 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177762 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177764 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177767 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177770 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177772 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177775 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177777 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177780 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177783 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177786 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177788 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177791 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177793 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177797 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177799 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 17:56:11.185363 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177802 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177804 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177808 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177810 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177815 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177817 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177820 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177822 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177825 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177827 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177830 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177833 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177835 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177838 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177840 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177843 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177845 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177848 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177850 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177853 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 17:56:11.185838 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177855 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177858 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177860 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177863 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177866 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177868 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177871 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177873 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177877 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177881 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177885 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177888 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177891 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177893 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177896 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177899 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177904 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177923 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177927 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177931 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 17:56:11.186368 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177935 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177939 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177942 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177945 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177948 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177950 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177953 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177956 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177958 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177961 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177963 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177966 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177970 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177973 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177976 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177979 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177982 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177985 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177988 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 17:56:11.186847 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177990 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177993 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.177996 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178000 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178002 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178006 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178010 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178012 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178015 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178020 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178022 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.178025 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.178677 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.185531 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.185548 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 17:56:11.187335 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.185569 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.185575 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185637 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185643 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185646 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185650 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185652 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185655 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185658 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185661 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185664 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185666 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185669 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185672 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185674 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185677 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185680 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185682 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185685 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185687 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 17:56:11.187709 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185690 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185692 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185695 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185697 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185700 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185702 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185705 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185708 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185710 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185713 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185715 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185719 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185722 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185725 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185728 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185730 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185733 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185736 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185738 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185741 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 17:56:11.188172 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185743 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185746 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185748 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185751 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185754 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185756 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185759 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185761 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185764 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185766 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185769 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185772 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185774 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185777 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185780 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185782 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185785 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185787 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185790 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185792 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 17:56:11.188667 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185794 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185797 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185800 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185803 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185807 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185809 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185812 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185814 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185817 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185821 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185825 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185828 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185831 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185835 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185838 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185841 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185843 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185846 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185849 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 17:56:11.189138 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185852 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185854 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185857 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185860 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185862 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185865 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185868 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185870 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185873 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185877 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185879 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185882 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185885 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185887 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185890 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185892 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185895 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185898 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185901 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 17:56:11.189636 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185904 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185906 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.185909 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.185914 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186036 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186041 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186044 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186047 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186050 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186052 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186055 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186057 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186060 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186063 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186065 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 17:56:11.190079 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186068 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186071 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186073 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186076 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186078 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186081 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186083 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186086 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186089 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186092 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186095 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186098 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186100 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186103 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186105 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186108 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186121 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186125 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186128 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186131 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 17:56:11.190465 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186133 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186136 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186139 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186142 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186144 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186147 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186149 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186152 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186154 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186157 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186160 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186162 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186165 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186167 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186169 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186172 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186174 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186192 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186196 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186199 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 17:56:11.190937 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186202 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186205 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186207 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186210 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186212 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186215 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186217 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186220 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186223 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186226 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186229 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186232 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186236 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186239 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186242 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186245 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186248 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186251 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186253 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 17:56:11.191419 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186256 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186258 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186261 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186263 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186266 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186268 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186271 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186273 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186277 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186280 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186283 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186286 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186289 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186291 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186294 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186296 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186299 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186301 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186304 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 17:56:11.191862 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186307 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186310 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186312 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186314 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186317 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186320 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186323 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186325 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186328 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:11.186331 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.186336 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.186447 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.188523 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.189993 2567 server.go:1019] "Starting client certificate rotation" Jul 20 17:56:11.192336 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.190091 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 17:56:11.192681 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.190132 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 17:56:11.192681 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.192533 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 17:56:11.195810 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.195794 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 17:56:11.210535 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.210511 2567 log.go:25] "Validated CRI v1 runtime API" Jul 20 17:56:11.213356 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.213339 2567 log.go:25] "Validated CRI v1 image API" Jul 20 17:56:11.214286 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.214266 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 17:56:11.219213 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.219170 2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8907c841-b5ae-4a1a-ad27-9b7768ce0c6b:/dev/nvme0n1p3 eae96ef8-8f0a-4576-bd7c-e5497c35e5c4:/dev/nvme0n1p4] Jul 20 17:56:11.219284 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.219212 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 17:56:11.221718 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.221697 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 17:56:11.225855 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.225743 2567 manager.go:216] Machine: {Timestamp:2026-07-20 17:56:11.224516904 +0000 UTC m=+0.352634062 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3139520 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e7b058c0121521f45d5fb66a5b5fe SystemUUID:ec2e7b05-8c01-2152-1f45-d5fb66a5b5fe BootID:35d72d27-cd56-447a-a04a-3aa3e90c45a5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:5a:31:af:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:5a:31:af:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:fc:6d:42:db:4e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 17:56:11.225855 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.225847 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 17:56:11.225987 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.225968 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 17:56:11.229018 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.228980 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 17:56:11.229211 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.229023 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-135.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 17:56:11.230211 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.230199 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 17:56:11.230269 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.230217 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 17:56:11.230269 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.230228 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 17:56:11.230879 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.230867 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 17:56:11.232066 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.232054 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 17:56:11.236274 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236260 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 20 17:56:11.236412 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236401 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 17:56:11.236468 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236442 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 20 17:56:11.236468 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236457 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 17:56:11.236550 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236473 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 17:56:11.236550 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236488 2567 kubelet.go:398] "Adding apiserver pod source" Jul 20 17:56:11.236550 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236500 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 17:56:11.236892 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236879 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 17:56:11.236977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.236904 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 17:56:11.238991 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.238973 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 17:56:11.240923 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.240901 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 17:56:11.240999 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.240945 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 17:56:11.242058 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242044 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 17:56:11.242134 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242070 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 17:56:11.242134 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242083 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 17:56:11.242134 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242094 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 17:56:11.242134 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242106 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 17:56:11.242134 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242123 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 17:56:11.242334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242138 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 17:56:11.242334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242148 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 17:56:11.242334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242158 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 17:56:11.242334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242171 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 17:56:11.242334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242200 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 17:56:11.242334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242209 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 17:56:11.242965 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242951 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 17:56:11.243049 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.242970 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 17:56:11.246092 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.246077 2567 server.go:1268] "Started kubelet" Jul 20 17:56:11.246417 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.246320 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 17:56:11.246494 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.246446 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 17:56:11.246688 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.246608 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 17:56:11.247001 ip-10-0-133-135 systemd[1]: Started Kubernetes Kubelet. Jul 20 17:56:11.247811 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.247607 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-135.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 17:56:11.247811 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.247657 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-135.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 17:56:11.247811 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.247800 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 17:56:11.248303 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.248283 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 17:56:11.250382 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.250365 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 20 17:56:11.251503 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.251477 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9hmzs" Jul 20 17:56:11.252977 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.252959 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 17:56:11.254408 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.253572 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-135.ec2.internal.18c4111d9de101a1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-135.ec2.internal,UID:ip-10-0-133-135.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-135.ec2.internal,},FirstTimestamp:2026-07-20 17:56:11.246051745 +0000 UTC m=+0.374168953,LastTimestamp:2026-07-20 17:56:11.246051745 +0000 UTC m=+0.374168953,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-135.ec2.internal,}" Jul 20 17:56:11.254408 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.254396 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 17:56:11.254570 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.254389 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 17:56:11.254570 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.254521 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.255034 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255011 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 17:56:11.255034 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255035 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 17:56:11.255172 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255052 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 17:56:11.255258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255214 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 20 17:56:11.255258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255224 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 20 17:56:11.255417 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255401 2567 factory.go:153] Registering CRI-O factory Jul 20 17:56:11.255501 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255421 2567 factory.go:223] Registration of the crio container factory successfully Jul 20 17:56:11.255501 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255475 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 17:56:11.255501 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255483 2567 factory.go:55] Registering systemd factory Jul 20 17:56:11.255501 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255489 2567 factory.go:223] Registration of the systemd container factory successfully Jul 20 17:56:11.255714 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255504 2567 factory.go:103] Registering Raw factory Jul 20 17:56:11.255714 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255514 2567 manager.go:1196] Started watching for new ooms in manager Jul 20 17:56:11.255882 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.255869 2567 manager.go:349] Starting recovery of all containers Jul 20 17:56:11.257678 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.257619 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-135.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 17:56:11.257797 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.257774 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 17:56:11.261764 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.261742 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9hmzs" Jul 20 17:56:11.265011 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.264994 2567 manager.go:354] Recovery completed Jul 20 17:56:11.270528 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.270516 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.272613 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.272594 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.272675 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.272646 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.272675 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.272665 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.273236 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.273217 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 17:56:11.273324 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.273237 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 17:56:11.273324 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.273258 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.273324 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.273281 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 20 17:56:11.274605 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.274588 2567 policy_none.go:49] "None policy: Start" Jul 20 17:56:11.274705 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.274612 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 17:56:11.274705 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.274626 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 17:56:11.274817 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.274762 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.274817 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.274791 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.274920 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.274819 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.275465 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.275382 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-135.ec2.internal.18c4111d9f7650a8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-135.ec2.internal,UID:ip-10-0-133-135.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-135.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-135.ec2.internal,},FirstTimestamp:2026-07-20 17:56:11.272614056 +0000 UTC m=+0.400731224,LastTimestamp:2026-07-20 17:56:11.272614056 +0000 UTC m=+0.400731224,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-135.ec2.internal,}" Jul 20 17:56:11.275987 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.275974 2567 policy_none.go:47] "Start" Jul 20 17:56:11.312066 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.311847 2567 manager.go:337] "Starting Device Plugin manager" Jul 20 17:56:11.312180 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.312099 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 17:56:11.312180 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.312116 2567 server.go:87] "Starting device plugin registration server" Jul 20 17:56:11.312424 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.312409 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 17:56:11.312507 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.312429 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 17:56:11.312584 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.312558 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 17:56:11.312690 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.312659 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 17:56:11.312690 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.312669 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 17:56:11.313302 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.313283 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 17:56:11.313395 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.313341 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.413455 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.413413 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.414760 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.414599 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.414760 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.414633 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.414760 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.414645 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.414760 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.414678 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.424122 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.424073 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.424122 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.424101 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-135.ec2.internal\": node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.432473 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.432451 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 17:56:11.433765 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.433731 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 17:56:11.433890 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.433778 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 17:56:11.433890 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.433802 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 17:56:11.433890 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.433846 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 17:56:11.436448 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.436425 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 17:56:11.474440 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.474411 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.534265 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.534225 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal"] Jul 20 17:56:11.534370 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.534323 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.535487 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.535469 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.535588 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.535504 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.535588 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.535520 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.536996 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.536982 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.537146 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537108 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.537185 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537152 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.537744 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537728 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.537815 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537750 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.537815 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537760 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.537978 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537967 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.538016 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537981 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.538016 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.537990 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.539307 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.539293 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.539361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.539318 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 17:56:11.540052 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.540039 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientMemory" Jul 20 17:56:11.540104 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.540059 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 17:56:11.540104 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.540068 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeHasSufficientPID" Jul 20 17:56:11.556516 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.556496 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/003bec3a3d41ca76f61aa5b879940780-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal\" (UID: \"003bec3a3d41ca76f61aa5b879940780\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.556616 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.556521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/003bec3a3d41ca76f61aa5b879940780-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal\" (UID: \"003bec3a3d41ca76f61aa5b879940780\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.556616 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.556544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6af33a66b3bbfc5fa2f8affa10cca68-config\") pod \"kube-apiserver-proxy-ip-10-0-133-135.ec2.internal\" (UID: \"e6af33a66b3bbfc5fa2f8affa10cca68\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.566022 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.566002 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-135.ec2.internal\" not found" node="ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.569636 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.569617 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-135.ec2.internal\" not found" node="ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.574728 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.574702 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.657463 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.657431 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/003bec3a3d41ca76f61aa5b879940780-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal\" (UID: \"003bec3a3d41ca76f61aa5b879940780\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.657463 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.657469 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/003bec3a3d41ca76f61aa5b879940780-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal\" (UID: \"003bec3a3d41ca76f61aa5b879940780\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.657621 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.657485 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6af33a66b3bbfc5fa2f8affa10cca68-config\") pod \"kube-apiserver-proxy-ip-10-0-133-135.ec2.internal\" (UID: \"e6af33a66b3bbfc5fa2f8affa10cca68\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.657621 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.657543 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/003bec3a3d41ca76f61aa5b879940780-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal\" (UID: \"003bec3a3d41ca76f61aa5b879940780\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.657621 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.657595 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6af33a66b3bbfc5fa2f8affa10cca68-config\") pod \"kube-apiserver-proxy-ip-10-0-133-135.ec2.internal\" (UID: \"e6af33a66b3bbfc5fa2f8affa10cca68\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.657721 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.657630 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/003bec3a3d41ca76f61aa5b879940780-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal\" (UID: \"003bec3a3d41ca76f61aa5b879940780\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.675172 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.675114 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.775283 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.775247 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.870004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.869820 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.872436 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:11.872411 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" Jul 20 17:56:11.876426 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.876407 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:11.977270 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:11.977145 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.078456 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.078404 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.174593 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.174563 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 17:56:12.178819 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.178801 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.189838 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.189804 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 17:56:12.189973 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.189954 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 17:56:12.190019 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.189999 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 17:56:12.266165 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.266069 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 17:51:12 +0000 UTC" deadline="2028-01-28 05:49:42.725615108 +0000 UTC" Jul 20 17:56:12.266165 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.266101 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13355h53m30.459517318s" Jul 20 17:56:12.278889 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.278866 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.379781 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.379751 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.472558 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:12.472514 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6af33a66b3bbfc5fa2f8affa10cca68.slice/crio-debe53f6598b02a3f2c1d0b0d7f681655c44d56056386361be8b13c860023646 WatchSource:0}: Error finding container debe53f6598b02a3f2c1d0b0d7f681655c44d56056386361be8b13c860023646: Status 404 returned error can't find the container with id debe53f6598b02a3f2c1d0b0d7f681655c44d56056386361be8b13c860023646 Jul 20 17:56:12.474393 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:12.474358 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod003bec3a3d41ca76f61aa5b879940780.slice/crio-c3efca062ff6bec6ae98e0adce05ebdbe7420b5788e89fcae34aef3d6c0b8d09 WatchSource:0}: Error finding container c3efca062ff6bec6ae98e0adce05ebdbe7420b5788e89fcae34aef3d6c0b8d09: Status 404 returned error can't find the container with id c3efca062ff6bec6ae98e0adce05ebdbe7420b5788e89fcae34aef3d6c0b8d09 Jul 20 17:56:12.478644 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.478629 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 17:56:12.480442 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.480415 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.581486 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.581390 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.682837 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:12.682795 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-135.ec2.internal\" not found" Jul 20 17:56:12.710396 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.710369 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 17:56:12.751301 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.751270 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 17:56:12.755688 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.755668 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" Jul 20 17:56:12.768086 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.768062 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 17:56:12.768955 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.768941 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" Jul 20 17:56:12.778005 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:12.777985 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 17:56:13.237877 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.237841 2567 apiserver.go:52] "Watching apiserver" Jul 20 17:56:13.247962 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.247933 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 17:56:13.248463 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.248432 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-r2mhk","openshift-ovn-kubernetes/ovnkube-node-b8hvn","kube-system/global-pull-secret-syncer-5c4v2","kube-system/konnectivity-agent-llkf2","kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal","openshift-cluster-node-tuning-operator/tuned-x7jlg","openshift-dns/node-resolver-vjrf4","openshift-image-registry/node-ca-xrnks","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal","openshift-network-operator/iptables-alerter-vvqdr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r","openshift-multus/multus-additional-cni-plugins-2tbm4","openshift-multus/multus-gvwhz","openshift-multus/network-metrics-daemon-jlzvz"] Jul 20 17:56:13.249963 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.249942 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.251602 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.251581 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.252898 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.252876 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.253003 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.252970 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lmvvp\"" Jul 20 17:56:13.253003 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.252988 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:13.253189 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.253168 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.253314 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.253293 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.254154 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.254129 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p7mp5\"" Jul 20 17:56:13.254255 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.254220 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.254374 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.254353 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 17:56:13.254438 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.254386 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.254574 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.254561 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.255028 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.255010 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 17:56:13.255721 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.255605 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.255721 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.255684 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:13.257000 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.256975 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.257000 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.256985 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 17:56:13.257476 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.257459 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dkn9l\"" Jul 20 17:56:13.257553 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.257498 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 17:56:13.258622 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.258346 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.259287 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.259263 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.259664 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.259643 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6m6xv\"" Jul 20 17:56:13.259768 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.259734 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.259841 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.259804 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.260997 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.260971 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jl7wq\"" Jul 20 17:56:13.261117 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.261016 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 17:56:13.261117 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.261043 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.261117 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.260975 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.261484 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.261466 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.262161 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.262145 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 17:56:13.262331 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.262316 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nf987\"" Jul 20 17:56:13.262402 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.262368 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.262484 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.262467 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 17:56:13.263006 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.262985 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.263130 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.263108 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.264498 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264476 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.264598 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264531 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 17:56:13.264598 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264576 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 17:56:13.264717 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264606 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6jj8h\"" Jul 20 17:56:13.264774 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264741 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 17:56:13.264858 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264822 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.264969 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.264915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:13.265036 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.264993 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:13.265417 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.265285 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 17:56:13.265898 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.265738 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 17:56:13.265898 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.265759 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 17:56:13.265898 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.265749 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 17:56:13.265898 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.265779 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mn7cb\"" Jul 20 17:56:13.266788 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.266766 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.267665 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267641 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-socket-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.267757 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267682 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.267757 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267712 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.267757 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwj7x\" (UniqueName: \"kubernetes.io/projected/77543f6c-716c-4da5-871f-dca1a0db25b8-kube-api-access-jwj7x\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.267891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267771 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx22s\" (UniqueName: \"kubernetes.io/projected/b9e88ee1-f628-49d6-a148-63f4d2482d0a-kube-api-access-gx22s\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.267891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267800 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02060614-726f-4116-8956-25ad464aa2c6-tmp-dir\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.267891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.267891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267853 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-device-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.267891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267882 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-sys-fs\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267946 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02060614-726f-4116-8956-25ad464aa2c6-hosts-file\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.267974 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-registration-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.268018 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b8386c3-4a92-46e9-9ac6-b93d336c0ca2-agent-certs\") pod \"konnectivity-agent-llkf2\" (UID: \"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2\") " pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.268049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b8386c3-4a92-46e9-9ac6-b93d336c0ca2-konnectivity-ca\") pod \"konnectivity-agent-llkf2\" (UID: \"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2\") " pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.268075 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f8584624-316d-4b6e-9463-3ead0ce5b732-kubelet-config\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.268106 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fd6p\" (UniqueName: \"kubernetes.io/projected/02060614-726f-4116-8956-25ad464aa2c6-kube-api-access-6fd6p\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.268205 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.268132 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f8584624-316d-4b6e-9463-3ead0ce5b732-dbus\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.273422 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.270497 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 17:56:13.273422 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.270597 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 17:56:13.273756 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.273722 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7hjbp\"" Jul 20 17:56:13.273847 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.273820 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 17:56:13.296347 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.296326 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h8bps" Jul 20 17:56:13.308039 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.308013 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h8bps" Jul 20 17:56:13.356838 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.356812 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 17:56:13.368341 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368312 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-socket-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwj7x\" (UniqueName: \"kubernetes.io/projected/77543f6c-716c-4da5-871f-dca1a0db25b8-kube-api-access-jwj7x\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.368474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-env-overrides\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.368474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368399 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7fcj\" (UniqueName: \"kubernetes.io/projected/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-kube-api-access-l7fcj\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.368474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02060614-726f-4116-8956-25ad464aa2c6-tmp-dir\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.368474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368484 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-socket-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368551 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368588 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-device-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-sys-fs\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368644 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.368696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368671 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-multus-certs\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368716 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-sys-fs\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368714 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-device-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368752 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02060614-726f-4116-8956-25ad464aa2c6-tmp-dir\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02060614-726f-4116-8956-25ad464aa2c6-hosts-file\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368823 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02060614-726f-4116-8956-25ad464aa2c6-hosts-file\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368833 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b8386c3-4a92-46e9-9ac6-b93d336c0ca2-konnectivity-ca\") pod \"konnectivity-agent-llkf2\" (UID: \"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2\") " pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368864 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-modprobe-d\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-var-lib-kubelet\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368928 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-k8s-cni-cncf-io\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.368941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-cni-bin\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.368990 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-hostroot\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369038 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7f78e87-a5a8-4b46-9933-8aec39656f7c-serviceca\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369077 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fd6p\" (UniqueName: \"kubernetes.io/projected/02060614-726f-4116-8956-25ad464aa2c6-kube-api-access-6fd6p\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369108 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f8584624-316d-4b6e-9463-3ead0ce5b732-dbus\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369146 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-kubelet\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369172 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-var-lib-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369198 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-etc-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369243 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-systemd\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-systemd\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-node-log\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-tmp\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369371 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-cnibin\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369390 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f8584624-316d-4b6e-9463-3ead0ce5b732-dbus\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.369415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369411 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-log-socket\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369440 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369469 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-ovnkube-config\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369505 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369530 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysconfig\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369559 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-registration-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b8386c3-4a92-46e9-9ac6-b93d336c0ca2-agent-certs\") pod \"konnectivity-agent-llkf2\" (UID: \"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2\") " pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369611 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-run-ovn-kubernetes\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369586 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b8386c3-4a92-46e9-9ac6-b93d336c0ca2-konnectivity-ca\") pod \"konnectivity-agent-llkf2\" (UID: \"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2\") " pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369638 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-ovnkube-script-lib\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369728 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66bvn\" (UniqueName: \"kubernetes.io/projected/a7f78e87-a5a8-4b46-9933-8aec39656f7c-kube-api-access-66bvn\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369727 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-registration-dir\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369800 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-lib-modules\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369841 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-socket-dir-parent\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369873 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f8584624-316d-4b6e-9463-3ead0ce5b732-kubelet-config\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369899 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dba577f3-59da-4331-8ac9-94811742621d-ovn-node-metrics-cert\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370051 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369928 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcbbq\" (UniqueName: \"kubernetes.io/projected/f3094be0-0efb-4507-ad8d-38bcf81c6561-kube-api-access-gcbbq\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369955 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-kubernetes\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369955 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f8584624-316d-4b6e-9463-3ead0ce5b732-kubelet-config\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.369979 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-tuned\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-system-cni-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370090 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370118 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3228b41-d8ff-4a80-8595-f726af002bc8-iptables-alerter-script\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370143 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-cnibin\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370176 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-run\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.370204 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370266 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-host\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.370322 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:13.87027576 +0000 UTC m=+2.998392910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370361 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx22s\" (UniqueName: \"kubernetes.io/projected/b9e88ee1-f628-49d6-a148-63f4d2482d0a-kube-api-access-gx22s\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370413 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-systemd-units\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370446 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-cni-bin\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-system-cni-dir\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.370834 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370501 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-conf-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glhtf\" (UniqueName: \"kubernetes.io/projected/e5a700d7-d24e-4e00-a050-c7553fd29039-kube-api-access-glhtf\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-ovn\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-cni-binary-copy\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370588 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysctl-d\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysctl-conf\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370643 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-sys\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-etc-kubernetes\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370723 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7f78e87-a5a8-4b46-9933-8aec39656f7c-host\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370747 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-run-netns\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370771 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmzq\" (UniqueName: \"kubernetes.io/projected/f3228b41-d8ff-4a80-8595-f726af002bc8-kube-api-access-qbmzq\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370792 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-cni-multus\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-daemon-config\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370920 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnm6h\" (UniqueName: \"kubernetes.io/projected/dba577f3-59da-4331-8ac9-94811742621d-kube-api-access-jnm6h\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.371562 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370942 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3228b41-d8ff-4a80-8595-f726af002bc8-host-slash\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.370991 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.370992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.371044 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:13.871026372 +0000 UTC m=+2.999143533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371086 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-cni-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371103 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9e88ee1-f628-49d6-a148-63f4d2482d0a-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371117 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-slash\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371217 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-os-release\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-os-release\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371282 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5a700d7-d24e-4e00-a050-c7553fd29039-cni-binary-copy\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-netns\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371350 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-kubelet\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.372176 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.371375 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-cni-netd\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.373908 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.373890 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b8386c3-4a92-46e9-9ac6-b93d336c0ca2-agent-certs\") pod \"konnectivity-agent-llkf2\" (UID: \"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2\") " pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.378371 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.378328 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fd6p\" (UniqueName: \"kubernetes.io/projected/02060614-726f-4116-8956-25ad464aa2c6-kube-api-access-6fd6p\") pod \"node-resolver-vjrf4\" (UID: \"02060614-726f-4116-8956-25ad464aa2c6\") " pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.378486 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.378443 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwj7x\" (UniqueName: \"kubernetes.io/projected/77543f6c-716c-4da5-871f-dca1a0db25b8-kube-api-access-jwj7x\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.382125 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.382099 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx22s\" (UniqueName: \"kubernetes.io/projected/b9e88ee1-f628-49d6-a148-63f4d2482d0a-kube-api-access-gx22s\") pod \"aws-ebs-csi-driver-node-h4w8r\" (UID: \"b9e88ee1-f628-49d6-a148-63f4d2482d0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.439096 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.439044 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" event={"ID":"e6af33a66b3bbfc5fa2f8affa10cca68","Type":"ContainerStarted","Data":"debe53f6598b02a3f2c1d0b0d7f681655c44d56056386361be8b13c860023646"} Jul 20 17:56:13.440491 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.440458 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" event={"ID":"003bec3a3d41ca76f61aa5b879940780","Type":"ContainerStarted","Data":"c3efca062ff6bec6ae98e0adce05ebdbe7420b5788e89fcae34aef3d6c0b8d09"} Jul 20 17:56:13.472041 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472004 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-var-lib-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472041 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472047 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-etc-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472070 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-systemd\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472096 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-systemd\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472116 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-node-log\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472138 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-tmp\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472143 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-etc-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472162 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-cnibin\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472184 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-log-socket\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472206 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-systemd\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472218 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-var-lib-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-node-log\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472153 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-systemd\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472258 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472208 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472282 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-log-socket\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.472313 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-ovnkube-config\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472331 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472348 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-cnibin\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472356 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysconfig\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472386 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-run-ovn-kubernetes\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472411 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-ovnkube-script-lib\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472475 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysconfig\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472495 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-run-ovn-kubernetes\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66bvn\" (UniqueName: \"kubernetes.io/projected/a7f78e87-a5a8-4b46-9933-8aec39656f7c-kube-api-access-66bvn\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472549 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-lib-modules\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472566 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-socket-dir-parent\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472589 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dba577f3-59da-4331-8ac9-94811742621d-ovn-node-metrics-cert\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcbbq\" (UniqueName: \"kubernetes.io/projected/f3094be0-0efb-4507-ad8d-38bcf81c6561-kube-api-access-gcbbq\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-kubernetes\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472703 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-tuned\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472729 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-system-cni-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3228b41-d8ff-4a80-8595-f726af002bc8-iptables-alerter-script\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.473074 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-cnibin\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472824 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-run\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-host\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-systemd-units\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472880 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-ovnkube-config\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472904 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-cni-bin\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-system-cni-dir\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472941 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-cnibin\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-lib-modules\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472955 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-conf-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472991 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-conf-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472995 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-ovnkube-script-lib\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473011 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glhtf\" (UniqueName: \"kubernetes.io/projected/e5a700d7-d24e-4e00-a050-c7553fd29039-kube-api-access-glhtf\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.472993 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-system-cni-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473047 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-run\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473066 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-cni-bin\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473079 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-systemd-units\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473115 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-system-cni-dir\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.474004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473142 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-kubernetes\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473143 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-host\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-ovn\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-cni-binary-copy\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473227 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-ovn\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473229 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473237 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-socket-dir-parent\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473381 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysctl-d\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473410 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysctl-conf\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-sys\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473460 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-etc-kubernetes\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473483 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7f78e87-a5a8-4b46-9933-8aec39656f7c-host\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-run-netns\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473535 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmzq\" (UniqueName: \"kubernetes.io/projected/f3228b41-d8ff-4a80-8595-f726af002bc8-kube-api-access-qbmzq\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473540 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3228b41-d8ff-4a80-8595-f726af002bc8-iptables-alerter-script\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473561 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysctl-d\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473562 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-cni-multus\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473603 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-etc-kubernetes\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.474958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473606 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-daemon-config\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473618 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-sys\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473646 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-cni-multus\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473656 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnm6h\" (UniqueName: \"kubernetes.io/projected/dba577f3-59da-4331-8ac9-94811742621d-kube-api-access-jnm6h\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3228b41-d8ff-4a80-8595-f726af002bc8-host-slash\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473708 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473731 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-cni-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473746 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-cni-binary-copy\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-slash\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473766 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-sysctl-conf\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473793 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-slash\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473815 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7f78e87-a5a8-4b46-9933-8aec39656f7c-host\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473830 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-os-release\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473829 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-run-openvswitch\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473854 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3228b41-d8ff-4a80-8595-f726af002bc8-host-slash\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473862 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-os-release\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-cni-dir\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.475953 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5a700d7-d24e-4e00-a050-c7553fd29039-cni-binary-copy\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473900 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-run-netns\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473939 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-netns\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-kubelet\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.473995 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-cni-netd\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-env-overrides\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7fcj\" (UniqueName: \"kubernetes.io/projected/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-kube-api-access-l7fcj\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474111 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-multus-certs\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474159 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-modprobe-d\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474182 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-var-lib-kubelet\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-k8s-cni-cncf-io\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474233 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-cni-bin\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-hostroot\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7f78e87-a5a8-4b46-9933-8aec39656f7c-serviceca\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474329 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-kubelet\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474402 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e5a700d7-d24e-4e00-a050-c7553fd29039-multus-daemon-config\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474417 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5a700d7-d24e-4e00-a050-c7553fd29039-cni-binary-copy\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.476911 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474418 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-kubelet\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474484 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474502 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-os-release\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-os-release\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474537 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-kubelet\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474543 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dba577f3-59da-4331-8ac9-94811742621d-host-cni-netd\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474560 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-var-lib-kubelet\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474583 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-netns\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474635 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-multus-certs\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474789 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3094be0-0efb-4507-ad8d-38bcf81c6561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474812 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-modprobe-d\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474833 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-var-lib-cni-bin\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474855 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-hostroot\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474863 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dba577f3-59da-4331-8ac9-94811742621d-env-overrides\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.474888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e5a700d7-d24e-4e00-a050-c7553fd29039-host-run-k8s-cni-cncf-io\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.475102 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-tmp\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.475269 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7f78e87-a5a8-4b46-9933-8aec39656f7c-serviceca\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.475398 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dba577f3-59da-4331-8ac9-94811742621d-ovn-node-metrics-cert\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.477891 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.475845 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3094be0-0efb-4507-ad8d-38bcf81c6561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.478825 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.476145 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-etc-tuned\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.486514 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.486462 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glhtf\" (UniqueName: \"kubernetes.io/projected/e5a700d7-d24e-4e00-a050-c7553fd29039-kube-api-access-glhtf\") pod \"multus-gvwhz\" (UID: \"e5a700d7-d24e-4e00-a050-c7553fd29039\") " pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.486514 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.486481 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnm6h\" (UniqueName: \"kubernetes.io/projected/dba577f3-59da-4331-8ac9-94811742621d-kube-api-access-jnm6h\") pod \"ovnkube-node-b8hvn\" (UID: \"dba577f3-59da-4331-8ac9-94811742621d\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.486716 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.486551 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmzq\" (UniqueName: \"kubernetes.io/projected/f3228b41-d8ff-4a80-8595-f726af002bc8-kube-api-access-qbmzq\") pod \"iptables-alerter-vvqdr\" (UID: \"f3228b41-d8ff-4a80-8595-f726af002bc8\") " pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.486716 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.486555 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66bvn\" (UniqueName: \"kubernetes.io/projected/a7f78e87-a5a8-4b46-9933-8aec39656f7c-kube-api-access-66bvn\") pod \"node-ca-xrnks\" (UID: \"a7f78e87-a5a8-4b46-9933-8aec39656f7c\") " pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.487679 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.487654 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcbbq\" (UniqueName: \"kubernetes.io/projected/f3094be0-0efb-4507-ad8d-38bcf81c6561-kube-api-access-gcbbq\") pod \"multus-additional-cni-plugins-2tbm4\" (UID: \"f3094be0-0efb-4507-ad8d-38bcf81c6561\") " pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.487809 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.487710 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:13.487809 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.487734 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:13.487809 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.487751 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:13.487979 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.487840 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:13.98782147 +0000 UTC m=+3.115938637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:13.487979 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.487870 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7fcj\" (UniqueName: \"kubernetes.io/projected/faa79f80-ebeb-43bf-9efb-9c1033bf1db4-kube-api-access-l7fcj\") pod \"tuned-x7jlg\" (UID: \"faa79f80-ebeb-43bf-9efb-9c1033bf1db4\") " pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.562465 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.562434 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vjrf4" Jul 20 17:56:13.570487 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.570464 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" Jul 20 17:56:13.579390 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.579368 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:13.587303 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.587278 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" Jul 20 17:56:13.594081 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.594057 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xrnks" Jul 20 17:56:13.603699 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.603672 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gvwhz" Jul 20 17:56:13.610505 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.610482 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:13.617158 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.617124 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vvqdr" Jul 20 17:56:13.623119 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.623100 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" Jul 20 17:56:13.736415 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.736380 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 17:56:13.877623 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.877521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:13.877805 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:13.877634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:13.877805 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.877691 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:13.877805 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.877723 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:13.877805 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.877777 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:14.877755789 +0000 UTC m=+4.005872949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:13.877805 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:13.877797 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:14.877787169 +0000 UTC m=+4.005904315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:13.982200 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.982138 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa79f80_ebeb_43bf_9efb_9c1033bf1db4.slice/crio-38055f37e79fa2268b534f67be5908b655f43713fcdb8ce2bcf263b20d6dc0f1 WatchSource:0}: Error finding container 38055f37e79fa2268b534f67be5908b655f43713fcdb8ce2bcf263b20d6dc0f1: Status 404 returned error can't find the container with id 38055f37e79fa2268b534f67be5908b655f43713fcdb8ce2bcf263b20d6dc0f1 Jul 20 17:56:13.986916 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.986883 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02060614_726f_4116_8956_25ad464aa2c6.slice/crio-e2e2a672aefa3d5010396b36c6fefa9fdb557bd9c665204fd66d76f663a84f0a WatchSource:0}: Error finding container e2e2a672aefa3d5010396b36c6fefa9fdb557bd9c665204fd66d76f663a84f0a: Status 404 returned error can't find the container with id e2e2a672aefa3d5010396b36c6fefa9fdb557bd9c665204fd66d76f663a84f0a Jul 20 17:56:13.988853 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.988831 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f78e87_a5a8_4b46_9933_8aec39656f7c.slice/crio-dc8999c43febf8894187c753e38ab3f9192f89447fb374662d7c36d7670b7117 WatchSource:0}: Error finding container dc8999c43febf8894187c753e38ab3f9192f89447fb374662d7c36d7670b7117: Status 404 returned error can't find the container with id dc8999c43febf8894187c753e38ab3f9192f89447fb374662d7c36d7670b7117 Jul 20 17:56:13.989523 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.989499 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e88ee1_f628_49d6_a148_63f4d2482d0a.slice/crio-fd6415319fc589073b5a77cbe02fc4600ded46b5f0ec43192a8853b6913f4cb3 WatchSource:0}: Error finding container fd6415319fc589073b5a77cbe02fc4600ded46b5f0ec43192a8853b6913f4cb3: Status 404 returned error can't find the container with id fd6415319fc589073b5a77cbe02fc4600ded46b5f0ec43192a8853b6913f4cb3 Jul 20 17:56:13.990455 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.990347 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3228b41_d8ff_4a80_8595_f726af002bc8.slice/crio-fabed0693981a355e1a6f39d583e1158d9a3cb5db093950752db8ab02cb7ddca WatchSource:0}: Error finding container fabed0693981a355e1a6f39d583e1158d9a3cb5db093950752db8ab02cb7ddca: Status 404 returned error can't find the container with id fabed0693981a355e1a6f39d583e1158d9a3cb5db093950752db8ab02cb7ddca Jul 20 17:56:13.991873 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.991844 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3094be0_0efb_4507_ad8d_38bcf81c6561.slice/crio-b00d4752203f6ded3e9b554de339675f924992bb0ed3ae26cd6aeb496206f2bf WatchSource:0}: Error finding container b00d4752203f6ded3e9b554de339675f924992bb0ed3ae26cd6aeb496206f2bf: Status 404 returned error can't find the container with id b00d4752203f6ded3e9b554de339675f924992bb0ed3ae26cd6aeb496206f2bf Jul 20 17:56:13.992915 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.992848 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a700d7_d24e_4e00_a050_c7553fd29039.slice/crio-3b975f2ebd0eb6b21a1f16b0b20527144d45b8228fc4530b79457a877ee1b0db WatchSource:0}: Error finding container 3b975f2ebd0eb6b21a1f16b0b20527144d45b8228fc4530b79457a877ee1b0db: Status 404 returned error can't find the container with id 3b975f2ebd0eb6b21a1f16b0b20527144d45b8228fc4530b79457a877ee1b0db Jul 20 17:56:13.994475 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.994429 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8386c3_4a92_46e9_9ac6_b93d336c0ca2.slice/crio-c698fdf4f65dd4b381ff4c42b0ae77d703c28f1256e84cfd176c5fd71478a1eb WatchSource:0}: Error finding container c698fdf4f65dd4b381ff4c42b0ae77d703c28f1256e84cfd176c5fd71478a1eb: Status 404 returned error can't find the container with id c698fdf4f65dd4b381ff4c42b0ae77d703c28f1256e84cfd176c5fd71478a1eb Jul 20 17:56:13.995534 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:13.995508 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba577f3_59da_4331_8ac9_94811742621d.slice/crio-f5a88a77dda44e38ab4f00579476e6b0fd4ffba8e6da114e1d0e8a42309d171f WatchSource:0}: Error finding container f5a88a77dda44e38ab4f00579476e6b0fd4ffba8e6da114e1d0e8a42309d171f: Status 404 returned error can't find the container with id f5a88a77dda44e38ab4f00579476e6b0fd4ffba8e6da114e1d0e8a42309d171f Jul 20 17:56:14.060353 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.060292 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 17:56:14.079091 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.079060 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:14.079275 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.079255 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:14.079341 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.079282 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:14.079341 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.079297 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:14.079419 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.079357 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:15.07934204 +0000 UTC m=+4.207459186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:14.310861 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.310756 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 17:51:14 +0000 UTC" deadline="2028-04-22 03:18:48.169178768 +0000 UTC" Jul 20 17:56:14.310861 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.310805 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15393h22m33.858376607s" Jul 20 17:56:14.435107 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.435074 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:14.435301 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.435244 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:14.445082 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.445043 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gvwhz" event={"ID":"e5a700d7-d24e-4e00-a050-c7553fd29039","Type":"ContainerStarted","Data":"3b975f2ebd0eb6b21a1f16b0b20527144d45b8228fc4530b79457a877ee1b0db"} Jul 20 17:56:14.446799 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.446762 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerStarted","Data":"b00d4752203f6ded3e9b554de339675f924992bb0ed3ae26cd6aeb496206f2bf"} Jul 20 17:56:14.448260 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.448224 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vvqdr" event={"ID":"f3228b41-d8ff-4a80-8595-f726af002bc8","Type":"ContainerStarted","Data":"fabed0693981a355e1a6f39d583e1158d9a3cb5db093950752db8ab02cb7ddca"} Jul 20 17:56:14.450008 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.449968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" event={"ID":"b9e88ee1-f628-49d6-a148-63f4d2482d0a","Type":"ContainerStarted","Data":"fd6415319fc589073b5a77cbe02fc4600ded46b5f0ec43192a8853b6913f4cb3"} Jul 20 17:56:14.451427 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.451393 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xrnks" event={"ID":"a7f78e87-a5a8-4b46-9933-8aec39656f7c","Type":"ContainerStarted","Data":"dc8999c43febf8894187c753e38ab3f9192f89447fb374662d7c36d7670b7117"} Jul 20 17:56:14.457428 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.457394 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" event={"ID":"e6af33a66b3bbfc5fa2f8affa10cca68","Type":"ContainerStarted","Data":"32ea40b072d01b582c5e52ada810a360a8048b2563c80566cb95ea073529a81a"} Jul 20 17:56:14.460229 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.460174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vjrf4" event={"ID":"02060614-726f-4116-8956-25ad464aa2c6","Type":"ContainerStarted","Data":"e2e2a672aefa3d5010396b36c6fefa9fdb557bd9c665204fd66d76f663a84f0a"} Jul 20 17:56:14.463321 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.463290 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" event={"ID":"faa79f80-ebeb-43bf-9efb-9c1033bf1db4","Type":"ContainerStarted","Data":"38055f37e79fa2268b534f67be5908b655f43713fcdb8ce2bcf263b20d6dc0f1"} Jul 20 17:56:14.465528 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.465499 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"f5a88a77dda44e38ab4f00579476e6b0fd4ffba8e6da114e1d0e8a42309d171f"} Jul 20 17:56:14.466989 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.466960 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-llkf2" event={"ID":"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2","Type":"ContainerStarted","Data":"c698fdf4f65dd4b381ff4c42b0ae77d703c28f1256e84cfd176c5fd71478a1eb"} Jul 20 17:56:14.889058 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.889012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:14.889287 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:14.889084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:14.889287 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.889268 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:14.889434 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.889337 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:16.88931528 +0000 UTC m=+6.017432431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:14.889785 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.889763 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:14.889858 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:14.889838 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:16.889818374 +0000 UTC m=+6.017935544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:15.090675 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:15.090629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:15.091418 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:15.090893 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:15.091418 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:15.090921 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:15.091418 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:15.090936 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:15.091418 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:15.090998 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:17.090977972 +0000 UTC m=+6.219095140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:15.312222 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:15.312067 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 17:51:14 +0000 UTC" deadline="2028-04-08 17:04:01.687595826 +0000 UTC" Jul 20 17:56:15.312222 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:15.312114 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15071h7m46.37548733s" Jul 20 17:56:15.437555 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:15.436707 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:15.437555 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:15.436854 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:15.437555 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:15.437331 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:15.437555 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:15.437450 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:16.434967 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:16.434640 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:16.435501 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:16.435044 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:16.497524 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:16.497474 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" event={"ID":"003bec3a3d41ca76f61aa5b879940780","Type":"ContainerStarted","Data":"ea878295981d278e4a946228766c3718eaafa782843fefcf3f11abc88f41af0c"} Jul 20 17:56:16.517464 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:16.517367 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-135.ec2.internal" podStartSLOduration=3.517343016 podStartE2EDuration="3.517343016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:14.496468589 +0000 UTC m=+3.624585776" watchObservedRunningTime="2026-07-20 17:56:16.517343016 +0000 UTC m=+5.645460186" Jul 20 17:56:16.910072 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:16.909981 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:16.910286 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:16.910082 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:16.910286 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:16.910232 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:16.910400 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:16.910301 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:20.910278912 +0000 UTC m=+10.038396062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:16.910768 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:16.910744 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:16.910873 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:16.910809 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:20.910792682 +0000 UTC m=+10.038909830 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:17.112181 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:17.112139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:17.112396 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:17.112342 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:17.112396 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:17.112364 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:17.112396 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:17.112377 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:17.112571 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:17.112438 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:21.112419034 +0000 UTC m=+10.240536183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:17.436908 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:17.436874 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:17.437396 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:17.437015 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:17.437473 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:17.437442 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:17.437604 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:17.437551 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:18.434043 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:18.434007 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:18.434301 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:18.434176 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:19.434580 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:19.434540 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:19.435077 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:19.434701 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:19.435257 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:19.435238 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:19.435379 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:19.435355 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:19.507651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:19.507560 2567 generic.go:356] "Generic (PLEG): container finished" podID="003bec3a3d41ca76f61aa5b879940780" containerID="ea878295981d278e4a946228766c3718eaafa782843fefcf3f11abc88f41af0c" exitCode=0 Jul 20 17:56:19.507651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:19.507626 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" event={"ID":"003bec3a3d41ca76f61aa5b879940780","Type":"ContainerDied","Data":"ea878295981d278e4a946228766c3718eaafa782843fefcf3f11abc88f41af0c"} Jul 20 17:56:20.434079 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:20.434045 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:20.434313 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:20.434187 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:20.948920 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:20.948803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:20.949437 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:20.949007 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:20.949437 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:20.949086 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:28.949062432 +0000 UTC m=+18.077179594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:20.949437 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:20.949114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:20.949437 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:20.949243 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:20.949437 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:20.949291 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:28.949277404 +0000 UTC m=+18.077394555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:21.151259 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:21.151215 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:21.151947 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:21.151472 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:21.151947 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:21.151501 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:21.151947 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:21.151515 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:21.151947 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:21.151577 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:29.151556636 +0000 UTC m=+18.279673800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:21.435572 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:21.435470 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:21.435780 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:21.435606 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:21.435780 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:21.435658 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:21.435902 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:21.435811 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:22.434231 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:22.434170 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:22.434742 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:22.434345 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:23.435267 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:23.435217 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:23.435636 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:23.435366 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:23.435636 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:23.435367 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:23.435636 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:23.435445 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:24.435016 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:24.434978 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:24.435203 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:24.435114 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:25.434229 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:25.434105 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:25.434229 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:25.434118 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:25.434701 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:25.434280 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:25.434701 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:25.434426 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:26.434928 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:26.434892 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:26.435432 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:26.435022 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:27.434326 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:27.434288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:27.434527 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:27.434298 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:27.434527 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:27.434437 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:27.434527 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:27.434513 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:28.434021 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:28.433986 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:28.434572 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:28.434108 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:29.015455 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:29.015421 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:29.015627 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:29.015473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:29.015627 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.015598 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:29.015734 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.015659 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:45.015643485 +0000 UTC m=+34.143760630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:29.015734 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.015596 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:29.015734 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.015693 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:45.015686524 +0000 UTC m=+34.143803668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:29.217299 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:29.217211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:29.217480 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.217409 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:29.217480 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.217434 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:29.217480 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.217447 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:29.217641 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.217519 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:45.217495354 +0000 UTC m=+34.345616964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:29.434021 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:29.433979 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:29.434545 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.434133 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:29.434545 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:29.434197 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:29.434545 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:29.434312 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:30.434666 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:30.434624 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:30.435093 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:30.434804 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:31.435137 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:31.435097 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:31.435616 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:31.435259 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:31.435722 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:31.435703 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:31.435869 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:31.435849 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:32.434307 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:32.434276 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:32.434487 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:32.434390 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:33.434862 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.434643 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:33.435531 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:33.434965 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:33.435531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.434690 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:33.435531 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:33.435107 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:33.536582 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.536547 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" event={"ID":"003bec3a3d41ca76f61aa5b879940780","Type":"ContainerStarted","Data":"00a679d7da6b4f4d6d6bc4ae9c2e8cb06d66ebe550a3823f43c10f18e833e1f3"} Jul 20 17:56:33.538873 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.538842 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"d858053aadd86c27d05ffc095799f4288cdca459a75dddfc374816e5070b690c"} Jul 20 17:56:33.539076 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.538939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"2bb80db2e049bdb75536b300325fe04b8969a9d3789de9f28d10e62c11464ac4"} Jul 20 17:56:33.542200 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.539425 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"621ff49d7a59034a72c09175607b12f6278857a03258a8e8bbc591d7f913d746"} Jul 20 17:56:33.543926 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.543873 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-llkf2" event={"ID":"9b8386c3-4a92-46e9-9ac6-b93d336c0ca2","Type":"ContainerStarted","Data":"b1fe2c14d7add01f0bf4be50205ac9de3d78a12d3a50b5a57196ef3d00c3481a"} Jul 20 17:56:33.545322 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.545300 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gvwhz" event={"ID":"e5a700d7-d24e-4e00-a050-c7553fd29039","Type":"ContainerStarted","Data":"94797e6790d62eff6c2d974c74010a7ccb72c1c5b4b97bc4639c44da006cc23a"} Jul 20 17:56:33.546941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.546863 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3094be0-0efb-4507-ad8d-38bcf81c6561" containerID="dc0cb28d31547c74e11913feaa1a7772e1af438fbede7c10e0e7da1c3a8e736a" exitCode=0 Jul 20 17:56:33.546941 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.546887 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerDied","Data":"dc0cb28d31547c74e11913feaa1a7772e1af438fbede7c10e0e7da1c3a8e736a"} Jul 20 17:56:33.548638 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.548616 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" event={"ID":"b9e88ee1-f628-49d6-a148-63f4d2482d0a","Type":"ContainerStarted","Data":"a00e63823b8e45d2267a2ac86dc404de91247393a230e8852eeaf0f2ec064494"} Jul 20 17:56:33.550250 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.550199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xrnks" event={"ID":"a7f78e87-a5a8-4b46-9933-8aec39656f7c","Type":"ContainerStarted","Data":"3f7434f84f3692a352e69ea1eb9ab91dd828a8c2d61a3e636b7299d61c29b233"} Jul 20 17:56:33.551659 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.551612 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-135.ec2.internal" podStartSLOduration=20.551598951 podStartE2EDuration="20.551598951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:33.551421596 +0000 UTC m=+22.679538764" watchObservedRunningTime="2026-07-20 17:56:33.551598951 +0000 UTC m=+22.679716118" Jul 20 17:56:33.551757 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.551634 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vjrf4" event={"ID":"02060614-726f-4116-8956-25ad464aa2c6","Type":"ContainerStarted","Data":"40e70a32f038711306d0724e1526b3709427f945d94447ff20ef96c8191ec87e"} Jul 20 17:56:33.553194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.553159 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" event={"ID":"faa79f80-ebeb-43bf-9efb-9c1033bf1db4","Type":"ContainerStarted","Data":"484d0445a433a9bc7b2f79b68b3f453b05720e90516f05fb7cf28007e88d3c59"} Jul 20 17:56:33.566136 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.566096 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-llkf2" podStartSLOduration=2.4167738930000002 podStartE2EDuration="21.566083215s" totalImagesPullingTime="19.149309322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:33.565420349 +0000 UTC m=+22.693537514" watchObservedRunningTime="2026-07-20 17:56:33.566083215 +0000 UTC m=+22.694200381" Jul 20 17:56:33.582086 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.582046 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gvwhz" podStartSLOduration=2.361951876 podStartE2EDuration="21.582029966s" totalImagesPullingTime="19.22007809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:33.581444396 +0000 UTC m=+22.709561564" watchObservedRunningTime="2026-07-20 17:56:33.582029966 +0000 UTC m=+22.710147132" Jul 20 17:56:33.615511 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.615470 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xrnks" podStartSLOduration=6.796928629 podStartE2EDuration="21.615456814s" totalImagesPullingTime="14.818528185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:33.615331495 +0000 UTC m=+22.743448661" watchObservedRunningTime="2026-07-20 17:56:33.615456814 +0000 UTC m=+22.743573980" Jul 20 17:56:33.647320 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:33.647219 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vjrf4" podStartSLOduration=6.827593623 podStartE2EDuration="21.647172765s" totalImagesPullingTime="14.819579142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:33.629002312 +0000 UTC m=+22.757119479" watchObservedRunningTime="2026-07-20 17:56:33.647172765 +0000 UTC m=+22.775289932" Jul 20 17:56:34.435018 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.434949 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:34.435377 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:34.435056 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:34.557766 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.557729 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"dbd7fa57573424e89bbe2b5ac4bfa167ebe106c1f2e7e2029928254453957a45"} Jul 20 17:56:34.557945 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.557814 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"735682e7ab7023322c2f2be047550f8eca253afadbd997a84af8e086a6967843"} Jul 20 17:56:34.557945 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.557857 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"f09a6ea4384386295f375cf6377972f001e239185a2e4f3eb3b44005970b6821"} Jul 20 17:56:34.559027 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.558999 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vvqdr" event={"ID":"f3228b41-d8ff-4a80-8595-f726af002bc8","Type":"ContainerStarted","Data":"2fcaa61cfd43dc00d01a0ff3d4efb17fb5e8796fca4156eb1cb5b6d380f3c796"} Jul 20 17:56:34.573076 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.573032 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x7jlg" podStartSLOduration=3.409478855 podStartE2EDuration="22.573016571s" totalImagesPullingTime="19.163537716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:33.65009358 +0000 UTC m=+22.778210747" watchObservedRunningTime="2026-07-20 17:56:34.573016571 +0000 UTC m=+23.701133776" Jul 20 17:56:34.573250 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.573227 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vvqdr" podStartSLOduration=3.419815015 podStartE2EDuration="22.573220442s" totalImagesPullingTime="19.153405427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:34.572846547 +0000 UTC m=+23.700963713" watchObservedRunningTime="2026-07-20 17:56:34.573220442 +0000 UTC m=+23.701337610" Jul 20 17:56:34.858027 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:34.858000 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 17:56:35.328375 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:35.328250 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T17:56:34.85802265Z","UUID":"3f2ddd52-ed01-4753-a005-776973f80839","Handler":null,"Name":"","Endpoint":""} Jul 20 17:56:35.331296 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:35.331263 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 17:56:35.331432 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:35.331314 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 17:56:35.434591 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:35.434557 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:35.434766 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:35.434690 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:35.434832 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:35.434766 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:35.434892 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:35.434855 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:35.563728 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:35.563688 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" event={"ID":"b9e88ee1-f628-49d6-a148-63f4d2482d0a","Type":"ContainerStarted","Data":"7cbe8e522c664c68d62ebc9cefcba6834b2f33f4e26620184ea1e8897b9e8716"} Jul 20 17:56:36.434410 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:36.433990 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:36.434410 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:36.434117 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:36.570278 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:36.570088 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"02687b03951368d3901c341026bb2d8835452aaf84651f44142d08d71c03dfdd"} Jul 20 17:56:37.435174 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:37.435087 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:37.435404 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:37.435245 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:37.435404 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:37.435320 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:37.435520 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:37.435448 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:37.846708 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:37.846625 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:37.847348 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:37.847330 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:38.434833 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.434809 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:38.434953 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:38.434930 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:38.577981 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.577719 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" event={"ID":"b9e88ee1-f628-49d6-a148-63f4d2482d0a","Type":"ContainerStarted","Data":"3785a934308a16c2645bd7beb512c371bbde5ba3ce69cfbd5c40f3b9e0199083"} Jul 20 17:56:38.582884 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.582853 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" event={"ID":"dba577f3-59da-4331-8ac9-94811742621d","Type":"ContainerStarted","Data":"bcfb16b11fffb5f0c1345283a466b61eaa625fadd2c8827d50838fccecc7d1ff"} Jul 20 17:56:38.583118 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.583094 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:38.583621 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.583600 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-llkf2" Jul 20 17:56:38.629910 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.629863 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" podStartSLOduration=7.408204168 podStartE2EDuration="26.629847489s" totalImagesPullingTime="19.221643321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:38.629415885 +0000 UTC m=+27.757533050" watchObservedRunningTime="2026-07-20 17:56:38.629847489 +0000 UTC m=+27.757964654" Jul 20 17:56:38.630287 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:38.630264 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4w8r" podStartSLOduration=2.717881846 podStartE2EDuration="26.630257057s" totalImagesPullingTime="23.912375211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:38.598618583 +0000 UTC m=+27.726735765" watchObservedRunningTime="2026-07-20 17:56:38.630257057 +0000 UTC m=+27.758374222" Jul 20 17:56:39.434736 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.434659 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:39.435526 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.434661 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:39.435526 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:39.434792 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:39.435526 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:39.434884 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:39.587890 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.587861 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3094be0-0efb-4507-ad8d-38bcf81c6561" containerID="c2feb500645a68193b8a4183fe16e856284076e25a051f4566133ae046ddf769" exitCode=0 Jul 20 17:56:39.588094 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.587962 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerDied","Data":"c2feb500645a68193b8a4183fe16e856284076e25a051f4566133ae046ddf769"} Jul 20 17:56:39.588094 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.588033 2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 20 17:56:39.590271 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.588711 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:39.590271 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.588784 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:39.611258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.608487 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:39.611258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:39.608892 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:40.347015 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.346979 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5c4v2"] Jul 20 17:56:40.347248 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.347121 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:40.347248 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:40.347221 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:40.350514 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.350478 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r2mhk"] Jul 20 17:56:40.350640 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.350596 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:40.350720 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:40.350696 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:40.351106 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.351081 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jlzvz"] Jul 20 17:56:40.351231 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.351207 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:40.351335 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:40.351316 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:40.603662 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:40.603630 2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 20 17:56:41.594373 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:41.594174 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3094be0-0efb-4507-ad8d-38bcf81c6561" containerID="41e3dd596ae1bfee6176a8f649e7b9232dab8795f13acfa4aa1fab92458b33ad" exitCode=0 Jul 20 17:56:41.594593 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:41.594215 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerDied","Data":"41e3dd596ae1bfee6176a8f649e7b9232dab8795f13acfa4aa1fab92458b33ad"} Jul 20 17:56:41.594593 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:41.594508 2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 20 17:56:42.435078 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:42.435002 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:42.435852 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:42.435001 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:42.435852 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:42.435142 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:42.435852 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:42.435247 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:42.435852 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:42.435004 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:42.435852 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:42.435428 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:43.235027 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:43.234939 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:56:43.235200 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:43.235160 2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 20 17:56:43.248280 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:43.248243 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" podUID="dba577f3-59da-4331-8ac9-94811742621d" containerName="ovnkube-controller" probeResult="failure" output="" Jul 20 17:56:43.256514 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:43.256481 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" podUID="dba577f3-59da-4331-8ac9-94811742621d" containerName="ovnkube-controller" probeResult="failure" output="" Jul 20 17:56:43.601109 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:43.601014 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3094be0-0efb-4507-ad8d-38bcf81c6561" containerID="3749a75f90e4da44c5bdc51092e22d3ff8dcdbb398b2cff0a54c1727e247ce20" exitCode=0 Jul 20 17:56:43.601109 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:43.601078 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerDied","Data":"3749a75f90e4da44c5bdc51092e22d3ff8dcdbb398b2cff0a54c1727e247ce20"} Jul 20 17:56:44.434723 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:44.434632 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:44.434723 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:44.434662 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:44.434723 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:44.434663 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:44.435000 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:44.434768 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5c4v2" podUID="f8584624-316d-4b6e-9463-3ead0ce5b732" Jul 20 17:56:44.435000 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:44.434910 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2mhk" podUID="852eb60a-f6aa-4217-b52c-7613703c3055" Jul 20 17:56:44.435109 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:44.435020 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlzvz" podUID="77543f6c-716c-4da5-871f-dca1a0db25b8" Jul 20 17:56:45.027727 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:45.027689 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:45.027727 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:45.027737 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:45.028304 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.027842 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:45.028304 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.027870 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:45.028304 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.027894 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:17.027879929 +0000 UTC m=+66.155997073 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 17:56:45.028304 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.027942 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret podName:f8584624-316d-4b6e-9463-3ead0ce5b732 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:17.02792327 +0000 UTC m=+66.156040434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret") pod "global-pull-secret-syncer-5c4v2" (UID: "f8584624-316d-4b6e-9463-3ead0ce5b732") : object "kube-system"/"original-pull-secret" not registered Jul 20 17:56:45.228874 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:45.228839 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:45.229070 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.228984 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 17:56:45.229070 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.229011 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 17:56:45.229070 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.229022 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hhgm6 for pod openshift-network-diagnostics/network-check-target-r2mhk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:45.229198 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:45.229083 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6 podName:852eb60a-f6aa-4217-b52c-7613703c3055 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:17.229061841 +0000 UTC m=+66.357179001 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hhgm6" (UniqueName: "kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6") pod "network-check-target-r2mhk" (UID: "852eb60a-f6aa-4217-b52c-7613703c3055") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 17:56:46.149175 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.149102 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-135.ec2.internal" event="NodeReady" Jul 20 17:56:46.149596 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.149277 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 17:56:46.194249 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.194215 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pvwbr"] Jul 20 17:56:46.196582 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.196547 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.196739 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.196618 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bkx8q"] Jul 20 17:56:46.198246 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.198228 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.199438 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.199418 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 17:56:46.199438 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.199430 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m6nkv\"" Jul 20 17:56:46.199569 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.199417 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 17:56:46.200932 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.200906 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 17:56:46.201193 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.201058 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 17:56:46.201193 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.201142 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 17:56:46.201684 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.201640 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9mph2\"" Jul 20 17:56:46.208545 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.208484 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvwbr"] Jul 20 17:56:46.211535 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.211511 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bkx8q"] Jul 20 17:56:46.335527 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.335487 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwp7c\" (UniqueName: \"kubernetes.io/projected/5882388a-c3ea-48a4-bed7-db61cd700c12-kube-api-access-hwp7c\") pod \"ingress-canary-bkx8q\" (UID: \"5882388a-c3ea-48a4-bed7-db61cd700c12\") " pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.335704 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.335560 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.335704 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.335589 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5882388a-c3ea-48a4-bed7-db61cd700c12-cert\") pod \"ingress-canary-bkx8q\" (UID: \"5882388a-c3ea-48a4-bed7-db61cd700c12\") " pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.335704 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.335647 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45dc5f22-b96b-457c-b060-a655c6b8adf0-config-volume\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.335704 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.335688 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc5f22-b96b-457c-b060-a655c6b8adf0-tmp-dir\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.335904 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.335714 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbkp5\" (UniqueName: \"kubernetes.io/projected/45dc5f22-b96b-457c-b060-a655c6b8adf0-kube-api-access-lbkp5\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.435058 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.434982 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:56:46.435058 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.435013 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:56:46.435058 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.435040 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:56:46.436245 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc5f22-b96b-457c-b060-a655c6b8adf0-tmp-dir\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.436333 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbkp5\" (UniqueName: \"kubernetes.io/projected/45dc5f22-b96b-457c-b060-a655c6b8adf0-kube-api-access-lbkp5\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.436333 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436278 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwp7c\" (UniqueName: \"kubernetes.io/projected/5882388a-c3ea-48a4-bed7-db61cd700c12-kube-api-access-hwp7c\") pod \"ingress-canary-bkx8q\" (UID: \"5882388a-c3ea-48a4-bed7-db61cd700c12\") " pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.436333 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.436443 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436347 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5882388a-c3ea-48a4-bed7-db61cd700c12-cert\") pod \"ingress-canary-bkx8q\" (UID: \"5882388a-c3ea-48a4-bed7-db61cd700c12\") " pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.436443 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45dc5f22-b96b-457c-b060-a655c6b8adf0-config-volume\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.436541 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:46.436460 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:56:46.436541 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:46.436521 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:46.936503198 +0000 UTC m=+36.064620342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:56:46.436652 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436622 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc5f22-b96b-457c-b060-a655c6b8adf0-tmp-dir\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.436890 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.436870 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45dc5f22-b96b-457c-b060-a655c6b8adf0-config-volume\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.438341 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.438317 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 17:56:46.438697 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.438680 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rwl6b\"" Jul 20 17:56:46.438766 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.438706 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 17:56:46.439016 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.439002 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 17:56:46.439108 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.439092 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nnhlg\"" Jul 20 17:56:46.440845 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.440826 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5882388a-c3ea-48a4-bed7-db61cd700c12-cert\") pod \"ingress-canary-bkx8q\" (UID: \"5882388a-c3ea-48a4-bed7-db61cd700c12\") " pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.442627 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.442602 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 17:56:46.450509 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.450374 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwp7c\" (UniqueName: \"kubernetes.io/projected/5882388a-c3ea-48a4-bed7-db61cd700c12-kube-api-access-hwp7c\") pod \"ingress-canary-bkx8q\" (UID: \"5882388a-c3ea-48a4-bed7-db61cd700c12\") " pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.450831 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.450814 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbkp5\" (UniqueName: \"kubernetes.io/projected/45dc5f22-b96b-457c-b060-a655c6b8adf0-kube-api-access-lbkp5\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.516157 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.516121 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bkx8q" Jul 20 17:56:46.675370 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.675322 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bkx8q"] Jul 20 17:56:46.680009 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:56:46.679915 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5882388a_c3ea_48a4_bed7_db61cd700c12.slice/crio-ac074026cb7af93c9c5c13d745dd2c6d5702ccbded20bdd8034182abf476e9f7 WatchSource:0}: Error finding container ac074026cb7af93c9c5c13d745dd2c6d5702ccbded20bdd8034182abf476e9f7: Status 404 returned error can't find the container with id ac074026cb7af93c9c5c13d745dd2c6d5702ccbded20bdd8034182abf476e9f7 Jul 20 17:56:46.940211 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:46.940160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:46.940346 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:46.940316 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:56:46.940397 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:46.940380 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:47.940364281 +0000 UTC m=+37.068481424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:56:47.613723 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:47.613675 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bkx8q" event={"ID":"5882388a-c3ea-48a4-bed7-db61cd700c12","Type":"ContainerStarted","Data":"ac074026cb7af93c9c5c13d745dd2c6d5702ccbded20bdd8034182abf476e9f7"} Jul 20 17:56:47.950113 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:47.950030 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:47.950280 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:47.950235 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:56:47.950337 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:47.950302 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:49.950282818 +0000 UTC m=+39.078399965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:56:49.967141 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:49.966948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:49.967141 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:49.967131 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:56:49.967579 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:49.967239 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:56:53.967216297 +0000 UTC m=+43.095333463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:56:50.622406 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:50.622371 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bkx8q" event={"ID":"5882388a-c3ea-48a4-bed7-db61cd700c12","Type":"ContainerStarted","Data":"21c1e6d972a114a9f9d1ee0fcab00f8cd35ab3615b0efc13ed7f48f6524f53ee"} Jul 20 17:56:50.625236 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:50.625206 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3094be0-0efb-4507-ad8d-38bcf81c6561" containerID="8e4e8107c71e80f4f77246da5796d0cf5946ff27e71309d00a6d5b4a9ddf677d" exitCode=0 Jul 20 17:56:50.625367 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:50.625258 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerDied","Data":"8e4e8107c71e80f4f77246da5796d0cf5946ff27e71309d00a6d5b4a9ddf677d"} Jul 20 17:56:50.639656 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:50.639613 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bkx8q" podStartSLOduration=1.6489188559999999 podStartE2EDuration="4.639598848s" totalImagesPullingTime="2.990679992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:50.639032057 +0000 UTC m=+39.767149224" watchObservedRunningTime="2026-07-20 17:56:50.639598848 +0000 UTC m=+39.767716014" Jul 20 17:56:51.630642 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:51.630605 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3094be0-0efb-4507-ad8d-38bcf81c6561" containerID="604858a93f1bb6f4156e6bbd608afe9da5dddf0c39ada54dd97f9329af6e2612" exitCode=0 Jul 20 17:56:51.631070 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:51.630693 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerDied","Data":"604858a93f1bb6f4156e6bbd608afe9da5dddf0c39ada54dd97f9329af6e2612"} Jul 20 17:56:52.635855 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:52.635816 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" event={"ID":"f3094be0-0efb-4507-ad8d-38bcf81c6561","Type":"ContainerStarted","Data":"26b3a74abb3796a705656a568de122b8266fecaa5c96cf144bbe816827e0b413"} Jul 20 17:56:52.660693 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:52.660643 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2tbm4" podStartSLOduration=8.198466491 podStartE2EDuration="40.660627015s" totalImagesPullingTime="32.462160524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:56:52.659242559 +0000 UTC m=+41.787359746" watchObservedRunningTime="2026-07-20 17:56:52.660627015 +0000 UTC m=+41.788744182" Jul 20 17:56:53.999786 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:56:53.999743 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:56:54.000213 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:53.999870 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:56:54.000213 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:56:53.999923 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:01.99991041 +0000 UTC m=+51.128027554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:57:02.058270 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:02.058228 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:57:02.058738 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:02.058379 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:57:02.058738 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:02.058443 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:18.058427753 +0000 UTC m=+67.186544914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:57:13.257434 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:13.257405 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8hvn" Jul 20 17:57:17.062515 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.062303 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:57:17.062997 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.062574 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:57:17.065615 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.065584 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 17:57:17.065769 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.065748 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 17:57:17.072745 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:17.072724 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 17:57:17.072816 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:17.072805 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs podName:77543f6c-716c-4da5-871f-dca1a0db25b8 nodeName:}" failed. No retries permitted until 2026-07-20 17:58:21.072785762 +0000 UTC m=+130.200902919 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs") pod "network-metrics-daemon-jlzvz" (UID: "77543f6c-716c-4da5-871f-dca1a0db25b8") : secret "metrics-daemon-secret" not found Jul 20 17:57:17.076444 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.076428 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f8584624-316d-4b6e-9463-3ead0ce5b732-original-pull-secret\") pod \"global-pull-secret-syncer-5c4v2\" (UID: \"f8584624-316d-4b6e-9463-3ead0ce5b732\") " pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:57:17.263902 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.263867 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:57:17.266694 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.266676 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 17:57:17.277468 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.277449 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 17:57:17.287958 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.287930 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhgm6\" (UniqueName: \"kubernetes.io/projected/852eb60a-f6aa-4217-b52c-7613703c3055-kube-api-access-hhgm6\") pod \"network-check-target-r2mhk\" (UID: \"852eb60a-f6aa-4217-b52c-7613703c3055\") " pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:57:17.352066 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.351991 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5c4v2" Jul 20 17:57:17.371774 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.371742 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rwl6b\"" Jul 20 17:57:17.379717 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.379696 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:57:17.481987 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.481953 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5c4v2"] Jul 20 17:57:17.485139 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:17.485108 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8584624_316d_4b6e_9463_3ead0ce5b732.slice/crio-feeac54bb25f69232e8aba7fa409a5f5f8367cb7923260634f5751af1109c7c8 WatchSource:0}: Error finding container feeac54bb25f69232e8aba7fa409a5f5f8367cb7923260634f5751af1109c7c8: Status 404 returned error can't find the container with id feeac54bb25f69232e8aba7fa409a5f5f8367cb7923260634f5751af1109c7c8 Jul 20 17:57:17.510107 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.510079 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r2mhk"] Jul 20 17:57:17.512754 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:17.512722 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852eb60a_f6aa_4217_b52c_7613703c3055.slice/crio-36782545ec10cd8a02f8c93d1a339d05b82931e58bc17e50f195285bbb6f725b WatchSource:0}: Error finding container 36782545ec10cd8a02f8c93d1a339d05b82931e58bc17e50f195285bbb6f725b: Status 404 returned error can't find the container with id 36782545ec10cd8a02f8c93d1a339d05b82931e58bc17e50f195285bbb6f725b Jul 20 17:57:17.694157 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.694063 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r2mhk" event={"ID":"852eb60a-f6aa-4217-b52c-7613703c3055","Type":"ContainerStarted","Data":"36782545ec10cd8a02f8c93d1a339d05b82931e58bc17e50f195285bbb6f725b"} Jul 20 17:57:17.695072 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:17.695048 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5c4v2" event={"ID":"f8584624-316d-4b6e-9463-3ead0ce5b732","Type":"ContainerStarted","Data":"feeac54bb25f69232e8aba7fa409a5f5f8367cb7923260634f5751af1109c7c8"} Jul 20 17:57:18.070003 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:18.069959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:57:18.070509 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:18.070121 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:57:18.070509 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:18.070236 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:50.070212515 +0000 UTC m=+99.198329675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:57:22.709670 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:22.709631 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r2mhk" event={"ID":"852eb60a-f6aa-4217-b52c-7613703c3055","Type":"ContainerStarted","Data":"e302a71972ff01418aae45abdd6256ac9e4dee61241aa015e80ec030e55cc965"} Jul 20 17:57:22.710233 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:22.709731 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:57:22.711126 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:22.711098 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5c4v2" event={"ID":"f8584624-316d-4b6e-9463-3ead0ce5b732","Type":"ContainerStarted","Data":"5bf3a97d6d6a60b31d9a75d6d6543556065e80139eb690b677bd61771a48396a"} Jul 20 17:57:22.735758 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:22.735709 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r2mhk" podStartSLOduration=65.990565412 podStartE2EDuration="1m10.73569413s" totalImagesPullingTime="4.745128718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:22.735197015 +0000 UTC m=+71.863314199" watchObservedRunningTime="2026-07-20 17:57:22.73569413 +0000 UTC m=+71.863811345" Jul 20 17:57:22.767349 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:22.767298 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5c4v2" podStartSLOduration=65.991776165 podStartE2EDuration="1m10.76728663s" totalImagesPullingTime="4.775510465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:22.767118626 +0000 UTC m=+71.895235791" watchObservedRunningTime="2026-07-20 17:57:22.76728663 +0000 UTC m=+71.895403795" Jul 20 17:57:40.528833 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.528722 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-nkrsq"] Jul 20 17:57:40.531342 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.531326 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.534388 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.534365 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f6xhb\"" Jul 20 17:57:40.534519 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.534483 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 17:57:40.535260 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.535242 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 17:57:40.535394 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.535270 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 20 17:57:40.535565 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.535550 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 20 17:57:40.553883 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.553856 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-nkrsq"] Jul 20 17:57:40.564785 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.564759 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 20 17:57:40.624726 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.624695 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d275ccd4-1373-4b29-bb57-ca14e91580d9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.624726 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.624728 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d275ccd4-1373-4b29-bb57-ca14e91580d9-serving-cert\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.624919 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.624762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d275ccd4-1373-4b29-bb57-ca14e91580d9-service-ca-bundle\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.624919 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.624779 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d275ccd4-1373-4b29-bb57-ca14e91580d9-snapshots\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.624919 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.624840 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d275ccd4-1373-4b29-bb57-ca14e91580d9-tmp\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.624919 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.624883 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v96v9\" (UniqueName: \"kubernetes.io/projected/d275ccd4-1373-4b29-bb57-ca14e91580d9-kube-api-access-v96v9\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726243 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d275ccd4-1373-4b29-bb57-ca14e91580d9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726243 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726253 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d275ccd4-1373-4b29-bb57-ca14e91580d9-serving-cert\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726473 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726393 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d275ccd4-1373-4b29-bb57-ca14e91580d9-service-ca-bundle\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726473 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726427 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d275ccd4-1373-4b29-bb57-ca14e91580d9-snapshots\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726473 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726452 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d275ccd4-1373-4b29-bb57-ca14e91580d9-tmp\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726655 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v96v9\" (UniqueName: \"kubernetes.io/projected/d275ccd4-1373-4b29-bb57-ca14e91580d9-kube-api-access-v96v9\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.726988 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.726966 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d275ccd4-1373-4b29-bb57-ca14e91580d9-service-ca-bundle\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.727077 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.727026 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d275ccd4-1373-4b29-bb57-ca14e91580d9-tmp\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.727115 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.727079 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d275ccd4-1373-4b29-bb57-ca14e91580d9-snapshots\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.727285 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.727267 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d275ccd4-1373-4b29-bb57-ca14e91580d9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.728606 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.728591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d275ccd4-1373-4b29-bb57-ca14e91580d9-serving-cert\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.738094 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.738076 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v96v9\" (UniqueName: \"kubernetes.io/projected/d275ccd4-1373-4b29-bb57-ca14e91580d9-kube-api-access-v96v9\") pod \"insights-operator-555d9d7885-nkrsq\" (UID: \"d275ccd4-1373-4b29-bb57-ca14e91580d9\") " pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.840097 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.840067 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" Jul 20 17:57:40.959530 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:40.959491 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-nkrsq"] Jul 20 17:57:40.963578 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:40.963552 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd275ccd4_1373_4b29_bb57_ca14e91580d9.slice/crio-abe54fbd0c960be51d39d50c8ce6186be63c405c85491715bff25a99ca6cc21b WatchSource:0}: Error finding container abe54fbd0c960be51d39d50c8ce6186be63c405c85491715bff25a99ca6cc21b: Status 404 returned error can't find the container with id abe54fbd0c960be51d39d50c8ce6186be63c405c85491715bff25a99ca6cc21b Jul 20 17:57:41.757278 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:41.757239 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" event={"ID":"d275ccd4-1373-4b29-bb57-ca14e91580d9","Type":"ContainerStarted","Data":"abe54fbd0c960be51d39d50c8ce6186be63c405c85491715bff25a99ca6cc21b"} Jul 20 17:57:42.404965 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.404931 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82"] Jul 20 17:57:42.406696 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.406680 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" Jul 20 17:57:42.409037 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.409013 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 17:57:42.409289 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.409272 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 17:57:42.410057 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.410040 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hpghp\"" Jul 20 17:57:42.419113 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.419093 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82"] Jul 20 17:57:42.437064 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.437033 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4c7d\" (UniqueName: \"kubernetes.io/projected/ec33470c-21bf-40ba-9d16-45ca4f3bbcca-kube-api-access-g4c7d\") pod \"volume-data-source-validator-78cf4c6f6-bpf82\" (UID: \"ec33470c-21bf-40ba-9d16-45ca4f3bbcca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" Jul 20 17:57:42.537601 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.537564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4c7d\" (UniqueName: \"kubernetes.io/projected/ec33470c-21bf-40ba-9d16-45ca4f3bbcca-kube-api-access-g4c7d\") pod \"volume-data-source-validator-78cf4c6f6-bpf82\" (UID: \"ec33470c-21bf-40ba-9d16-45ca4f3bbcca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" Jul 20 17:57:42.553174 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.553141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4c7d\" (UniqueName: \"kubernetes.io/projected/ec33470c-21bf-40ba-9d16-45ca4f3bbcca-kube-api-access-g4c7d\") pod \"volume-data-source-validator-78cf4c6f6-bpf82\" (UID: \"ec33470c-21bf-40ba-9d16-45ca4f3bbcca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" Jul 20 17:57:42.715110 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.715036 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" Jul 20 17:57:42.844517 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:42.844486 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82"] Jul 20 17:57:42.847304 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:42.847277 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec33470c_21bf_40ba_9d16_45ca4f3bbcca.slice/crio-bb347dae7fa7a998265c0b68a499e826afd272de42e20d43f970f108cbe0b986 WatchSource:0}: Error finding container bb347dae7fa7a998265c0b68a499e826afd272de42e20d43f970f108cbe0b986: Status 404 returned error can't find the container with id bb347dae7fa7a998265c0b68a499e826afd272de42e20d43f970f108cbe0b986 Jul 20 17:57:43.458468 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.458431 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rl452"] Jul 20 17:57:43.460980 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.460954 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.466169 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.466143 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 17:57:43.466401 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.466383 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 17:57:43.466517 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.466467 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-44cq2\"" Jul 20 17:57:43.466575 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.466511 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 17:57:43.466830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.466810 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 17:57:43.479231 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.479169 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 17:57:43.482078 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.482059 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rl452"] Jul 20 17:57:43.547010 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.546985 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xv5t\" (UniqueName: \"kubernetes.io/projected/e5783726-84ea-4afd-a948-32168ca72d7d-kube-api-access-7xv5t\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.547123 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.547025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e5783726-84ea-4afd-a948-32168ca72d7d-config\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.547123 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.547055 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e5783726-84ea-4afd-a948-32168ca72d7d-trusted-ca\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.547240 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.547140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e5783726-84ea-4afd-a948-32168ca72d7d-serving-cert\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.648394 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.648357 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xv5t\" (UniqueName: \"kubernetes.io/projected/e5783726-84ea-4afd-a948-32168ca72d7d-kube-api-access-7xv5t\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.648394 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.648401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e5783726-84ea-4afd-a948-32168ca72d7d-config\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.648654 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.648421 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e5783726-84ea-4afd-a948-32168ca72d7d-trusted-ca\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.648654 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.648446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e5783726-84ea-4afd-a948-32168ca72d7d-serving-cert\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.649166 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.649137 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e5783726-84ea-4afd-a948-32168ca72d7d-config\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.649358 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.649339 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e5783726-84ea-4afd-a948-32168ca72d7d-trusted-ca\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.650719 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.650699 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e5783726-84ea-4afd-a948-32168ca72d7d-serving-cert\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.657154 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.657133 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xv5t\" (UniqueName: \"kubernetes.io/projected/e5783726-84ea-4afd-a948-32168ca72d7d-kube-api-access-7xv5t\") pod \"console-operator-565b7b569d-rl452\" (UID: \"e5783726-84ea-4afd-a948-32168ca72d7d\") " pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.765267 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.765174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" event={"ID":"d275ccd4-1373-4b29-bb57-ca14e91580d9","Type":"ContainerStarted","Data":"54d9e600218f4551d2bbe878b967a1d21cdb182c955ef37c1a7a902a75b13dd6"} Jul 20 17:57:43.766603 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.766561 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" event={"ID":"ec33470c-21bf-40ba-9d16-45ca4f3bbcca","Type":"ContainerStarted","Data":"bb347dae7fa7a998265c0b68a499e826afd272de42e20d43f970f108cbe0b986"} Jul 20 17:57:43.772307 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.772253 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:43.784843 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.784793 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" podStartSLOduration=1.240421729 podStartE2EDuration="3.784773876s" totalImagesPullingTime="2.544352147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:43.783749309 +0000 UTC m=+92.911866476" watchObservedRunningTime="2026-07-20 17:57:43.784773876 +0000 UTC m=+92.912891042" Jul 20 17:57:43.902293 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:43.902254 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rl452"] Jul 20 17:57:43.905250 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:43.905220 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5783726_84ea_4afd_a948_32168ca72d7d.slice/crio-02604fce8db316623cbdde606449dc9632b61d892a5da337f70e93e14722e332 WatchSource:0}: Error finding container 02604fce8db316623cbdde606449dc9632b61d892a5da337f70e93e14722e332: Status 404 returned error can't find the container with id 02604fce8db316623cbdde606449dc9632b61d892a5da337f70e93e14722e332 Jul 20 17:57:44.770922 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:44.770837 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rl452" event={"ID":"e5783726-84ea-4afd-a948-32168ca72d7d","Type":"ContainerStarted","Data":"02604fce8db316623cbdde606449dc9632b61d892a5da337f70e93e14722e332"} Jul 20 17:57:44.772554 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:44.772514 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" event={"ID":"ec33470c-21bf-40ba-9d16-45ca4f3bbcca","Type":"ContainerStarted","Data":"cadabaaaa55877c722a10005e9f010673f79c3d5423650b885e04872b09ff9d6"} Jul 20 17:57:44.789475 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:44.789425 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-bpf82" podStartSLOduration=1.228903758 podStartE2EDuration="2.789406886s" totalImagesPullingTime="1.560503128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:44.788821409 +0000 UTC m=+93.916938570" watchObservedRunningTime="2026-07-20 17:57:44.789406886 +0000 UTC m=+93.917524098" Jul 20 17:57:45.105139 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.105103 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ts7vs"] Jul 20 17:57:45.107587 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.107564 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.110730 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.110710 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 17:57:45.111070 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.111052 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 17:57:45.114326 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.114302 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-krpk2\"" Jul 20 17:57:45.129117 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.129045 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ts7vs"] Jul 20 17:57:45.160895 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.160862 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvdgr\" (UniqueName: \"kubernetes.io/projected/ed285dc0-4058-4afa-a620-42799e4245ed-kube-api-access-fvdgr\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.161041 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.160909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed285dc0-4058-4afa-a620-42799e4245ed-crio-socket\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.161041 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.161010 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed285dc0-4058-4afa-a620-42799e4245ed-data-volume\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.161137 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.161056 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.161137 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.161094 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262345 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262302 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvdgr\" (UniqueName: \"kubernetes.io/projected/ed285dc0-4058-4afa-a620-42799e4245ed-kube-api-access-fvdgr\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262505 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262360 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed285dc0-4058-4afa-a620-42799e4245ed-crio-socket\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262505 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262430 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed285dc0-4058-4afa-a620-42799e4245ed-data-volume\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262505 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262462 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262505 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262505 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262496 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed285dc0-4058-4afa-a620-42799e4245ed-crio-socket\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.262775 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.262582 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 17:57:45.262775 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.262602 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 17:57:45.262775 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.262654 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:45.762635125 +0000 UTC m=+94.890752270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : secret "insights-runtime-extractor-tls" not found Jul 20 17:57:45.262775 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.262672 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:45.762661891 +0000 UTC m=+94.890779040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : configmap "kube-rbac-proxy" not found Jul 20 17:57:45.262906 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.262858 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed285dc0-4058-4afa-a620-42799e4245ed-data-volume\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.271096 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.271065 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvdgr\" (UniqueName: \"kubernetes.io/projected/ed285dc0-4058-4afa-a620-42799e4245ed-kube-api-access-fvdgr\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.765477 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.765430 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.765477 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:45.765483 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:45.765721 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.765595 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 17:57:45.765721 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.765603 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 17:57:45.765721 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.765658 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:46.765642416 +0000 UTC m=+95.893759560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : configmap "kube-rbac-proxy" not found Jul 20 17:57:45.765721 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:45.765677 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:46.765666828 +0000 UTC m=+95.893783972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : secret "insights-runtime-extractor-tls" not found Jul 20 17:57:46.772791 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:46.772739 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:46.772791 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:46.772799 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:46.773259 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:46.772873 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 17:57:46.773259 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:46.772919 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 17:57:46.773259 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:46.772933 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:48.772919777 +0000 UTC m=+97.901036922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : configmap "kube-rbac-proxy" not found Jul 20 17:57:46.773259 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:46.772990 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:48.772972006 +0000 UTC m=+97.901089150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : secret "insights-runtime-extractor-tls" not found Jul 20 17:57:46.779661 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:46.779640 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/0.log" Jul 20 17:57:46.779783 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:46.779674 2567 generic.go:356] "Generic (PLEG): container finished" podID="e5783726-84ea-4afd-a948-32168ca72d7d" containerID="d651fc11558c92655f40ed12985e9b8bcbf9ea4905bbd6f8d1ba0037ae4707b3" exitCode=255 Jul 20 17:57:46.779783 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:46.779726 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rl452" event={"ID":"e5783726-84ea-4afd-a948-32168ca72d7d","Type":"ContainerDied","Data":"d651fc11558c92655f40ed12985e9b8bcbf9ea4905bbd6f8d1ba0037ae4707b3"} Jul 20 17:57:46.779933 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:46.779921 2567 scope.go:117] "RemoveContainer" containerID="d651fc11558c92655f40ed12985e9b8bcbf9ea4905bbd6f8d1ba0037ae4707b3" Jul 20 17:57:47.288421 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.288393 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vjrf4_02060614-726f-4116-8956-25ad464aa2c6/dns-node-resolver/0.log" Jul 20 17:57:47.420055 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.420018 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx"] Jul 20 17:57:47.422002 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.421985 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.425865 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.425844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 20 17:57:47.426114 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.426096 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 20 17:57:47.426193 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.426161 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 20 17:57:47.426788 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.426774 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hwsks\"" Jul 20 17:57:47.426833 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.426820 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 20 17:57:47.442954 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.442926 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx"] Jul 20 17:57:47.478308 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.478270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-666zm\" (UniqueName: \"kubernetes.io/projected/5463db92-b0e2-4bd1-b601-c8d9349830a1-kube-api-access-666zm\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.478474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.478328 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5463db92-b0e2-4bd1-b601-c8d9349830a1-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.478474 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.478388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5463db92-b0e2-4bd1-b601-c8d9349830a1-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.578889 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.578855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5463db92-b0e2-4bd1-b601-c8d9349830a1-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.579041 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.578901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5463db92-b0e2-4bd1-b601-c8d9349830a1-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.579041 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.578967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-666zm\" (UniqueName: \"kubernetes.io/projected/5463db92-b0e2-4bd1-b601-c8d9349830a1-kube-api-access-666zm\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.579528 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.579504 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5463db92-b0e2-4bd1-b601-c8d9349830a1-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.581004 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.580986 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5463db92-b0e2-4bd1-b601-c8d9349830a1-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.595357 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.595325 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-666zm\" (UniqueName: \"kubernetes.io/projected/5463db92-b0e2-4bd1-b601-c8d9349830a1-kube-api-access-666zm\") pod \"kube-storage-version-migrator-operator-7658d9b749-lvvhx\" (UID: \"5463db92-b0e2-4bd1-b601-c8d9349830a1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.730936 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.730896 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" Jul 20 17:57:47.785481 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.785453 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 17:57:47.786370 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.786342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/0.log" Jul 20 17:57:47.786486 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.786394 2567 generic.go:356] "Generic (PLEG): container finished" podID="e5783726-84ea-4afd-a948-32168ca72d7d" containerID="86db74597396468f9a4e186b8c847ffb8b72449ae2d7432cc58e4afc8793e3d1" exitCode=255 Jul 20 17:57:47.786486 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.786429 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rl452" event={"ID":"e5783726-84ea-4afd-a948-32168ca72d7d","Type":"ContainerDied","Data":"86db74597396468f9a4e186b8c847ffb8b72449ae2d7432cc58e4afc8793e3d1"} Jul 20 17:57:47.786486 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.786481 2567 scope.go:117] "RemoveContainer" containerID="d651fc11558c92655f40ed12985e9b8bcbf9ea4905bbd6f8d1ba0037ae4707b3" Jul 20 17:57:47.787028 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.787006 2567 scope.go:117] "RemoveContainer" containerID="86db74597396468f9a4e186b8c847ffb8b72449ae2d7432cc58e4afc8793e3d1" Jul 20 17:57:47.787596 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:47.787215 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rl452_openshift-console-operator(e5783726-84ea-4afd-a948-32168ca72d7d)\"" pod="openshift-console-operator/console-operator-565b7b569d-rl452" podUID="e5783726-84ea-4afd-a948-32168ca72d7d" Jul 20 17:57:47.853548 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:47.853475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx"] Jul 20 17:57:47.856389 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:47.856363 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5463db92_b0e2_4bd1_b601_c8d9349830a1.slice/crio-cb87d6364ab19eb9f854b61c5efb1ed5ff0cfc9c41c9aec6e46e97e52e0e47dc WatchSource:0}: Error finding container cb87d6364ab19eb9f854b61c5efb1ed5ff0cfc9c41c9aec6e46e97e52e0e47dc: Status 404 returned error can't find the container with id cb87d6364ab19eb9f854b61c5efb1ed5ff0cfc9c41c9aec6e46e97e52e0e47dc Jul 20 17:57:48.080715 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.080686 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xrnks_a7f78e87-a5a8-4b46-9933-8aec39656f7c/node-ca/0.log" Jul 20 17:57:48.680340 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.680309 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bkx8q_5882388a-c3ea-48a4-bed7-db61cd700c12/serve-healthcheck-canary/0.log" Jul 20 17:57:48.788072 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.788032 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:48.788559 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.788080 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:48.788559 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:48.788225 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 17:57:48.788559 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:48.788251 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 17:57:48.788559 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:48.788294 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:52.788277684 +0000 UTC m=+101.916394828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : secret "insights-runtime-extractor-tls" not found Jul 20 17:57:48.788559 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:48.788309 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:57:52.788302518 +0000 UTC m=+101.916419661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : configmap "kube-rbac-proxy" not found Jul 20 17:57:48.792097 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.792074 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 17:57:48.792498 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.792476 2567 scope.go:117] "RemoveContainer" containerID="86db74597396468f9a4e186b8c847ffb8b72449ae2d7432cc58e4afc8793e3d1" Jul 20 17:57:48.792687 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:48.792653 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rl452_openshift-console-operator(e5783726-84ea-4afd-a948-32168ca72d7d)\"" pod="openshift-console-operator/console-operator-565b7b569d-rl452" podUID="e5783726-84ea-4afd-a948-32168ca72d7d" Jul 20 17:57:48.793497 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:48.793472 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" event={"ID":"5463db92-b0e2-4bd1-b601-c8d9349830a1","Type":"ContainerStarted","Data":"cb87d6364ab19eb9f854b61c5efb1ed5ff0cfc9c41c9aec6e46e97e52e0e47dc"} Jul 20 17:57:49.424613 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.424575 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw"] Jul 20 17:57:49.426458 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.426441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.428932 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.428902 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 20 17:57:49.429058 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.428930 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 20 17:57:49.429192 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.429167 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 20 17:57:49.430033 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.430003 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 20 17:57:49.430033 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.430021 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6lpch\"" Jul 20 17:57:49.439991 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.439969 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw"] Jul 20 17:57:49.493742 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.493706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.493896 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.493808 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7f7t\" (UniqueName: \"kubernetes.io/projected/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-kube-api-access-h7f7t\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.493896 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.493883 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-config\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.594605 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.594576 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7f7t\" (UniqueName: \"kubernetes.io/projected/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-kube-api-access-h7f7t\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.594770 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.594635 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-config\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.594770 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.594676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.595282 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.595256 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-config\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.596928 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.596902 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.603730 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.603700 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7f7t\" (UniqueName: \"kubernetes.io/projected/8c8d84a0-d438-4540-91a6-7c7d7bd04b2d-kube-api-access-h7f7t\") pod \"service-ca-operator-5bcbc94c78-7blsw\" (UID: \"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:49.735359 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:49.735271 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" Jul 20 17:57:50.099224 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.099166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:57:50.099662 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:50.099287 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 17:57:50.099662 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:50.099379 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls podName:45dc5f22-b96b-457c-b060-a655c6b8adf0 nodeName:}" failed. No retries permitted until 2026-07-20 17:58:54.099363379 +0000 UTC m=+163.227480522 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls") pod "dns-default-pvwbr" (UID: "45dc5f22-b96b-457c-b060-a655c6b8adf0") : secret "dns-default-metrics-tls" not found Jul 20 17:57:50.238910 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.238877 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s"] Jul 20 17:57:50.240949 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.240933 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" Jul 20 17:57:50.244371 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.244312 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pjncr\"" Jul 20 17:57:50.246800 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.246776 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw"] Jul 20 17:57:50.251691 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:50.251652 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8d84a0_d438_4540_91a6_7c7d7bd04b2d.slice/crio-639d407156075fe1e3009f07713d869b49b3820121bf5a840bd8a96d6b3936e2 WatchSource:0}: Error finding container 639d407156075fe1e3009f07713d869b49b3820121bf5a840bd8a96d6b3936e2: Status 404 returned error can't find the container with id 639d407156075fe1e3009f07713d869b49b3820121bf5a840bd8a96d6b3936e2 Jul 20 17:57:50.270838 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.270816 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s"] Jul 20 17:57:50.301025 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.300995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcn77\" (UniqueName: \"kubernetes.io/projected/c9820750-c68f-44f7-9ac3-5199a94926bc-kube-api-access-fcn77\") pod \"network-check-source-7bc5b6bf85-cdk8s\" (UID: \"c9820750-c68f-44f7-9ac3-5199a94926bc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" Jul 20 17:57:50.401764 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.401682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcn77\" (UniqueName: \"kubernetes.io/projected/c9820750-c68f-44f7-9ac3-5199a94926bc-kube-api-access-fcn77\") pod \"network-check-source-7bc5b6bf85-cdk8s\" (UID: \"c9820750-c68f-44f7-9ac3-5199a94926bc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" Jul 20 17:57:50.411685 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.411660 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcn77\" (UniqueName: \"kubernetes.io/projected/c9820750-c68f-44f7-9ac3-5199a94926bc-kube-api-access-fcn77\") pod \"network-check-source-7bc5b6bf85-cdk8s\" (UID: \"c9820750-c68f-44f7-9ac3-5199a94926bc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" Jul 20 17:57:50.548858 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.548821 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75d765bb8c-h6vjr"] Jul 20 17:57:50.551338 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.551319 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" Jul 20 17:57:50.551407 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.551361 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.553806 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.553783 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 17:57:50.553898 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.553828 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 17:57:50.554209 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.554169 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dl9wc\"" Jul 20 17:57:50.554323 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.554269 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 17:57:50.559039 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.559019 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 17:57:50.563817 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.563779 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75d765bb8c-h6vjr"] Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603069 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a848c84a-cd5e-42bf-9f14-c367f9a9f627-ca-trust-extracted\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603117 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-image-registry-private-configuration\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-certificates\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603241 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-trusted-ca\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603265 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-bound-sa-token\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj77j\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-kube-api-access-wj77j\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603355 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.604194 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.603384 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-installation-pull-secrets\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.677115 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.677040 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s"] Jul 20 17:57:50.679837 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:57:50.679811 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9820750_c68f_44f7_9ac3_5199a94926bc.slice/crio-b355009529b450ca588bc374f7c22fed6d2301f973745c6719ab3cd66cb8d541 WatchSource:0}: Error finding container b355009529b450ca588bc374f7c22fed6d2301f973745c6719ab3cd66cb8d541: Status 404 returned error can't find the container with id b355009529b450ca588bc374f7c22fed6d2301f973745c6719ab3cd66cb8d541 Jul 20 17:57:50.704550 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704453 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-image-registry-private-configuration\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704550 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704540 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-certificates\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704666 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704599 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-trusted-ca\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704666 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704641 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-bound-sa-token\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704776 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj77j\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-kube-api-access-wj77j\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704825 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704876 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704839 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-installation-pull-secrets\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.704925 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.704885 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a848c84a-cd5e-42bf-9f14-c367f9a9f627-ca-trust-extracted\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.705529 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:50.705065 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 17:57:50.705529 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:50.705087 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d765bb8c-h6vjr: secret "image-registry-tls" not found Jul 20 17:57:50.705529 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:50.705136 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls podName:a848c84a-cd5e-42bf-9f14-c367f9a9f627 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:51.205118587 +0000 UTC m=+100.333235748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls") pod "image-registry-75d765bb8c-h6vjr" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627") : secret "image-registry-tls" not found Jul 20 17:57:50.705529 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.705387 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a848c84a-cd5e-42bf-9f14-c367f9a9f627-ca-trust-extracted\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.705849 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.705549 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-certificates\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.705849 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.705651 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-trusted-ca\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.707272 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.707250 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-image-registry-private-configuration\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.707497 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.707474 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-installation-pull-secrets\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.713939 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.713920 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-bound-sa-token\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.714509 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.714493 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj77j\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-kube-api-access-wj77j\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:50.806028 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.805973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" event={"ID":"c9820750-c68f-44f7-9ac3-5199a94926bc","Type":"ContainerStarted","Data":"92a94868e721db48c6fb9d148de50473c7f0c6ebf3bdddef4d8fe66e72a254e2"} Jul 20 17:57:50.806234 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.806066 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" event={"ID":"c9820750-c68f-44f7-9ac3-5199a94926bc","Type":"ContainerStarted","Data":"b355009529b450ca588bc374f7c22fed6d2301f973745c6719ab3cd66cb8d541"} Jul 20 17:57:50.807586 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.807560 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" event={"ID":"5463db92-b0e2-4bd1-b601-c8d9349830a1","Type":"ContainerStarted","Data":"02cb5bd2ec29d984e5d91a803738d0e602d45f2b0a5f6fe34e6e504eaef5b5a7"} Jul 20 17:57:50.808704 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.808681 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" event={"ID":"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d","Type":"ContainerStarted","Data":"639d407156075fe1e3009f07713d869b49b3820121bf5a840bd8a96d6b3936e2"} Jul 20 17:57:50.822926 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.822881 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cdk8s" podStartSLOduration=0.822868354 podStartE2EDuration="822.868354ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:50.822195796 +0000 UTC m=+99.950312988" watchObservedRunningTime="2026-07-20 17:57:50.822868354 +0000 UTC m=+99.950985520" Jul 20 17:57:50.842592 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:50.842541 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" podStartSLOduration=1.568256475 podStartE2EDuration="3.842522424s" totalImagesPullingTime="2.274265949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:50.841223517 +0000 UTC m=+99.969340686" watchObservedRunningTime="2026-07-20 17:57:50.842522424 +0000 UTC m=+99.970639592" Jul 20 17:57:51.208686 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:51.208646 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:51.209096 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:51.208815 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 17:57:51.209096 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:51.208838 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d765bb8c-h6vjr: secret "image-registry-tls" not found Jul 20 17:57:51.209096 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:51.208905 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls podName:a848c84a-cd5e-42bf-9f14-c367f9a9f627 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:52.20888568 +0000 UTC m=+101.337002824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls") pod "image-registry-75d765bb8c-h6vjr" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627") : secret "image-registry-tls" not found Jul 20 17:57:52.219755 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:52.219712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:52.220136 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.219859 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 17:57:52.220136 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.219878 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d765bb8c-h6vjr: secret "image-registry-tls" not found Jul 20 17:57:52.220136 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.219934 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls podName:a848c84a-cd5e-42bf-9f14-c367f9a9f627 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:54.219918939 +0000 UTC m=+103.348036090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls") pod "image-registry-75d765bb8c-h6vjr" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627") : secret "image-registry-tls" not found Jul 20 17:57:52.825169 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:52.825136 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:52.825341 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:52.825248 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:57:52.825341 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.825291 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 17:57:52.825430 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.825360 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:58:00.825340531 +0000 UTC m=+109.953457691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : secret "insights-runtime-extractor-tls" not found Jul 20 17:57:52.825430 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.825376 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 17:57:52.825498 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:52.825452 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:58:00.825435414 +0000 UTC m=+109.953552565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : configmap "kube-rbac-proxy" not found Jul 20 17:57:53.716600 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:53.716567 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r2mhk" Jul 20 17:57:53.772685 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:53.772654 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:53.772841 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:53.772718 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:57:53.773220 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:53.773203 2567 scope.go:117] "RemoveContainer" containerID="86db74597396468f9a4e186b8c847ffb8b72449ae2d7432cc58e4afc8793e3d1" Jul 20 17:57:53.773424 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:53.773401 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rl452_openshift-console-operator(e5783726-84ea-4afd-a948-32168ca72d7d)\"" pod="openshift-console-operator/console-operator-565b7b569d-rl452" podUID="e5783726-84ea-4afd-a948-32168ca72d7d" Jul 20 17:57:53.818640 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:53.818601 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" event={"ID":"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d","Type":"ContainerStarted","Data":"32ebcac1c9f129e47a9950117958cd4e106d8fb898efd225a38cfac4d20b9903"} Jul 20 17:57:53.837657 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:53.837612 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" podStartSLOduration=1.846842048 podStartE2EDuration="4.837591294s" totalImagesPullingTime="2.990749246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:57:53.836547468 +0000 UTC m=+102.964664635" watchObservedRunningTime="2026-07-20 17:57:53.837591294 +0000 UTC m=+102.965708460" Jul 20 17:57:54.236494 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:54.236446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:54.236681 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:54.236612 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 17:57:54.236681 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:54.236634 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d765bb8c-h6vjr: secret "image-registry-tls" not found Jul 20 17:57:54.236758 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:54.236689 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls podName:a848c84a-cd5e-42bf-9f14-c367f9a9f627 nodeName:}" failed. No retries permitted until 2026-07-20 17:57:58.236673461 +0000 UTC m=+107.364790605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls") pod "image-registry-75d765bb8c-h6vjr" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627") : secret "image-registry-tls" not found Jul 20 17:57:58.270330 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:57:58.270283 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:57:58.270827 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:58.270430 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 17:57:58.270827 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:58.270450 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75d765bb8c-h6vjr: secret "image-registry-tls" not found Jul 20 17:57:58.270827 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:57:58.270505 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls podName:a848c84a-cd5e-42bf-9f14-c367f9a9f627 nodeName:}" failed. No retries permitted until 2026-07-20 17:58:06.270490557 +0000 UTC m=+115.398607701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls") pod "image-registry-75d765bb8c-h6vjr" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627") : secret "image-registry-tls" not found Jul 20 17:58:00.889742 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:00.889705 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:58:00.890160 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:00.889765 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:58:00.890160 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:00.889857 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 17:58:00.890160 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:00.889926 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls podName:ed285dc0-4058-4afa-a620-42799e4245ed nodeName:}" failed. No retries permitted until 2026-07-20 17:58:16.889909179 +0000 UTC m=+126.018026322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ts7vs" (UID: "ed285dc0-4058-4afa-a620-42799e4245ed") : secret "insights-runtime-extractor-tls" not found Jul 20 17:58:00.890307 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:00.890282 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed285dc0-4058-4afa-a620-42799e4245ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:58:05.435397 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:05.435359 2567 scope.go:117] "RemoveContainer" containerID="86db74597396468f9a4e186b8c847ffb8b72449ae2d7432cc58e4afc8793e3d1" Jul 20 17:58:05.858734 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:05.858704 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 17:58:05.858897 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:05.858772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rl452" event={"ID":"e5783726-84ea-4afd-a948-32168ca72d7d","Type":"ContainerStarted","Data":"44d410376a16825aae211f1dedb7a8705c6d16639a2c0d10a638338fafca5da9"} Jul 20 17:58:05.859049 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:05.859028 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:58:05.869619 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:05.869591 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-rl452" Jul 20 17:58:05.881059 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:05.879454 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-rl452" podStartSLOduration=20.625814586 podStartE2EDuration="22.87943464s" totalImagesPullingTime="2.253620054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:05.879156743 +0000 UTC m=+115.007273940" watchObservedRunningTime="2026-07-20 17:58:05.87943464 +0000 UTC m=+115.007551806" Jul 20 17:58:06.334110 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.334072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:58:06.336506 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.336477 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"image-registry-75d765bb8c-h6vjr\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:58:06.462745 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.462704 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:58:06.591948 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.591863 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75d765bb8c-h6vjr"] Jul 20 17:58:06.594601 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:06.594570 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda848c84a_cd5e_42bf_9f14_c367f9a9f627.slice/crio-b658e974ab631f9e25feb5f1a9a4043c105b32b486334e2ee92192d71d020114 WatchSource:0}: Error finding container b658e974ab631f9e25feb5f1a9a4043c105b32b486334e2ee92192d71d020114: Status 404 returned error can't find the container with id b658e974ab631f9e25feb5f1a9a4043c105b32b486334e2ee92192d71d020114 Jul 20 17:58:06.863869 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.863769 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" event={"ID":"a848c84a-cd5e-42bf-9f14-c367f9a9f627","Type":"ContainerStarted","Data":"f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1"} Jul 20 17:58:06.863869 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.863847 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" event={"ID":"a848c84a-cd5e-42bf-9f14-c367f9a9f627","Type":"ContainerStarted","Data":"b658e974ab631f9e25feb5f1a9a4043c105b32b486334e2ee92192d71d020114"} Jul 20 17:58:06.883496 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:06.883442 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" podStartSLOduration=16.883425527 podStartE2EDuration="16.883425527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:06.882031778 +0000 UTC m=+116.010148945" watchObservedRunningTime="2026-07-20 17:58:06.883425527 +0000 UTC m=+116.011542693" Jul 20 17:58:07.867522 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:07.867491 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:58:16.919689 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:16.919645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:58:16.922115 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:16.922089 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed285dc0-4058-4afa-a620-42799e4245ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ts7vs\" (UID: \"ed285dc0-4058-4afa-a620-42799e4245ed\") " pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:58:17.221838 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:17.221754 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-krpk2\"" Jul 20 17:58:17.229090 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:17.229071 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ts7vs" Jul 20 17:58:17.400869 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:17.400794 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ts7vs"] Jul 20 17:58:17.403665 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:17.403637 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded285dc0_4058_4afa_a620_42799e4245ed.slice/crio-8401a8757db2e3d390481d029943a5b4b96dacbc274096a81adddaa11fdee2c3 WatchSource:0}: Error finding container 8401a8757db2e3d390481d029943a5b4b96dacbc274096a81adddaa11fdee2c3: Status 404 returned error can't find the container with id 8401a8757db2e3d390481d029943a5b4b96dacbc274096a81adddaa11fdee2c3 Jul 20 17:58:17.898303 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:17.898264 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ts7vs" event={"ID":"ed285dc0-4058-4afa-a620-42799e4245ed","Type":"ContainerStarted","Data":"8401a8757db2e3d390481d029943a5b4b96dacbc274096a81adddaa11fdee2c3"} Jul 20 17:58:21.159587 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.159540 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:58:21.162087 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.162057 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77543f6c-716c-4da5-871f-dca1a0db25b8-metrics-certs\") pod \"network-metrics-daemon-jlzvz\" (UID: \"77543f6c-716c-4da5-871f-dca1a0db25b8\") " pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:58:21.265701 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.265670 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nnhlg\"" Jul 20 17:58:21.273343 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.273310 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlzvz" Jul 20 17:58:21.416659 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.414855 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh"] Jul 20 17:58:21.420780 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.420753 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.424088 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.424054 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 20 17:58:21.424236 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.424104 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 20 17:58:21.424540 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.424505 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 20 17:58:21.424749 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.424735 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-chcnk\"" Jul 20 17:58:21.425895 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.425874 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 20 17:58:21.433321 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.433297 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh"] Jul 20 17:58:21.461871 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.461814 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9598a58c-3f0b-4c79-8f0c-958df137ef19-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6479546b4b-2fzxh\" (UID: \"9598a58c-3f0b-4c79-8f0c-958df137ef19\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.462433 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.462161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b88n\" (UniqueName: \"kubernetes.io/projected/9598a58c-3f0b-4c79-8f0c-958df137ef19-kube-api-access-8b88n\") pod \"managed-serviceaccount-addon-agent-6479546b4b-2fzxh\" (UID: \"9598a58c-3f0b-4c79-8f0c-958df137ef19\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.477132 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.477098 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75d765bb8c-h6vjr"] Jul 20 17:58:21.482751 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.482708 2567 patch_prober.go:28] interesting pod/image-registry-75d765bb8c-h6vjr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 17:58:21.482890 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.482771 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:58:21.523124 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.523092 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4"] Jul 20 17:58:21.532606 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.532582 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:21.536633 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.536612 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 20 17:58:21.537338 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.537318 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-z9z66\"" Jul 20 17:58:21.547899 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.547876 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4"] Jul 20 17:58:21.562489 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.562466 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9598a58c-3f0b-4c79-8f0c-958df137ef19-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6479546b4b-2fzxh\" (UID: \"9598a58c-3f0b-4c79-8f0c-958df137ef19\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.562640 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.562506 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3c94574e-4dc8-4b76-b552-2bc13e9d25c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hwmq4\" (UID: \"3c94574e-4dc8-4b76-b552-2bc13e9d25c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:21.562640 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.562547 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b88n\" (UniqueName: \"kubernetes.io/projected/9598a58c-3f0b-4c79-8f0c-958df137ef19-kube-api-access-8b88n\") pod \"managed-serviceaccount-addon-agent-6479546b4b-2fzxh\" (UID: \"9598a58c-3f0b-4c79-8f0c-958df137ef19\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.565033 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.565007 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9598a58c-3f0b-4c79-8f0c-958df137ef19-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6479546b4b-2fzxh\" (UID: \"9598a58c-3f0b-4c79-8f0c-958df137ef19\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.593354 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.593323 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b88n\" (UniqueName: \"kubernetes.io/projected/9598a58c-3f0b-4c79-8f0c-958df137ef19-kube-api-access-8b88n\") pod \"managed-serviceaccount-addon-agent-6479546b4b-2fzxh\" (UID: \"9598a58c-3f0b-4c79-8f0c-958df137ef19\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.614352 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.614324 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6989456fd7-mfs4w"] Jul 20 17:58:21.616491 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.616475 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.647814 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.647785 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6989456fd7-mfs4w"] Jul 20 17:58:21.663729 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-registry-tls\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.663729 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663737 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-bound-sa-token\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.663977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663782 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c393437a-06b7-4663-a90f-ef4417c866d1-ca-trust-extracted\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.663977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jznfr\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-kube-api-access-jznfr\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.663977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663841 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c393437a-06b7-4663-a90f-ef4417c866d1-trusted-ca\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.663977 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c393437a-06b7-4663-a90f-ef4417c866d1-installation-pull-secrets\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.664243 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.663979 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3c94574e-4dc8-4b76-b552-2bc13e9d25c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hwmq4\" (UID: \"3c94574e-4dc8-4b76-b552-2bc13e9d25c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:21.664243 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.664015 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c393437a-06b7-4663-a90f-ef4417c866d1-image-registry-private-configuration\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.664243 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.664042 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c393437a-06b7-4663-a90f-ef4417c866d1-registry-certificates\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.666527 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.666505 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3c94574e-4dc8-4b76-b552-2bc13e9d25c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hwmq4\" (UID: \"3c94574e-4dc8-4b76-b552-2bc13e9d25c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:21.739671 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.739598 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" Jul 20 17:58:21.765308 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765277 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c393437a-06b7-4663-a90f-ef4417c866d1-ca-trust-extracted\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765308 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jznfr\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-kube-api-access-jznfr\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765337 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c393437a-06b7-4663-a90f-ef4417c866d1-trusted-ca\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765409 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c393437a-06b7-4663-a90f-ef4417c866d1-installation-pull-secrets\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c393437a-06b7-4663-a90f-ef4417c866d1-image-registry-private-configuration\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765471 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c393437a-06b7-4663-a90f-ef4417c866d1-registry-certificates\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765497 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-registry-tls\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-bound-sa-token\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.765822 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.765724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c393437a-06b7-4663-a90f-ef4417c866d1-ca-trust-extracted\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.766423 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.766395 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c393437a-06b7-4663-a90f-ef4417c866d1-registry-certificates\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.766531 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.766422 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c393437a-06b7-4663-a90f-ef4417c866d1-trusted-ca\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.768278 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.768253 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c393437a-06b7-4663-a90f-ef4417c866d1-installation-pull-secrets\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.768653 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.768633 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c393437a-06b7-4663-a90f-ef4417c866d1-image-registry-private-configuration\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.768743 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.768657 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-registry-tls\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.787882 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.787858 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-bound-sa-token\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.788168 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.788146 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jznfr\" (UniqueName: \"kubernetes.io/projected/c393437a-06b7-4663-a90f-ef4417c866d1-kube-api-access-jznfr\") pod \"image-registry-6989456fd7-mfs4w\" (UID: \"c393437a-06b7-4663-a90f-ef4417c866d1\") " pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:21.841254 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.841221 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:21.927595 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:21.927559 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:22.481236 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.481120 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jlzvz"] Jul 20 17:58:22.483658 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:22.483634 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77543f6c_716c_4da5_871f_dca1a0db25b8.slice/crio-4f8c2dbbbb08a89cae3ef08b0547be6b672b1b6b401732c6f9b48e2bd5688462 WatchSource:0}: Error finding container 4f8c2dbbbb08a89cae3ef08b0547be6b672b1b6b401732c6f9b48e2bd5688462: Status 404 returned error can't find the container with id 4f8c2dbbbb08a89cae3ef08b0547be6b672b1b6b401732c6f9b48e2bd5688462 Jul 20 17:58:22.708392 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.708363 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4"] Jul 20 17:58:22.712349 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:22.712323 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c94574e_4dc8_4b76_b552_2bc13e9d25c7.slice/crio-eb7a36e7f4f15857f9a6cad53fcba22b82c7d4e60b5e0a58a8f93419e71a35c7 WatchSource:0}: Error finding container eb7a36e7f4f15857f9a6cad53fcba22b82c7d4e60b5e0a58a8f93419e71a35c7: Status 404 returned error can't find the container with id eb7a36e7f4f15857f9a6cad53fcba22b82c7d4e60b5e0a58a8f93419e71a35c7 Jul 20 17:58:22.716949 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.716928 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh"] Jul 20 17:58:22.719931 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:22.719903 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9598a58c_3f0b_4c79_8f0c_958df137ef19.slice/crio-5cc8ed2625c1236c023a548efe2af7430fecf199db4a0b42bcdfc2b6266ba146 WatchSource:0}: Error finding container 5cc8ed2625c1236c023a548efe2af7430fecf199db4a0b42bcdfc2b6266ba146: Status 404 returned error can't find the container with id 5cc8ed2625c1236c023a548efe2af7430fecf199db4a0b42bcdfc2b6266ba146 Jul 20 17:58:22.722288 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.722267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6989456fd7-mfs4w"] Jul 20 17:58:22.722762 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:22.722742 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc393437a_06b7_4663_a90f_ef4417c866d1.slice/crio-c71737d017784a911f2528205a317532ddaa695e50cfa2d8587762a6ab7026c7 WatchSource:0}: Error finding container c71737d017784a911f2528205a317532ddaa695e50cfa2d8587762a6ab7026c7: Status 404 returned error can't find the container with id c71737d017784a911f2528205a317532ddaa695e50cfa2d8587762a6ab7026c7 Jul 20 17:58:22.921207 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.921154 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" event={"ID":"9598a58c-3f0b-4c79-8f0c-958df137ef19","Type":"ContainerStarted","Data":"5cc8ed2625c1236c023a548efe2af7430fecf199db4a0b42bcdfc2b6266ba146"} Jul 20 17:58:22.923116 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.923077 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" event={"ID":"3c94574e-4dc8-4b76-b552-2bc13e9d25c7","Type":"ContainerStarted","Data":"eb7a36e7f4f15857f9a6cad53fcba22b82c7d4e60b5e0a58a8f93419e71a35c7"} Jul 20 17:58:22.925389 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.925355 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ts7vs" event={"ID":"ed285dc0-4058-4afa-a620-42799e4245ed","Type":"ContainerStarted","Data":"6a801744d36aa7e4373b9562facae6c4d846e8db837457c42a8a8d5e41b93e31"} Jul 20 17:58:22.927882 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.927757 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" event={"ID":"c393437a-06b7-4663-a90f-ef4417c866d1","Type":"ContainerStarted","Data":"dbad28f153fca8195f4d4839b459e1adee0c1061cbdc0de5ed68c036eb446ccd"} Jul 20 17:58:22.927882 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.927808 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" event={"ID":"c393437a-06b7-4663-a90f-ef4417c866d1","Type":"ContainerStarted","Data":"c71737d017784a911f2528205a317532ddaa695e50cfa2d8587762a6ab7026c7"} Jul 20 17:58:22.927882 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.927851 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:22.929436 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.929411 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jlzvz" event={"ID":"77543f6c-716c-4da5-871f-dca1a0db25b8","Type":"ContainerStarted","Data":"4f8c2dbbbb08a89cae3ef08b0547be6b672b1b6b401732c6f9b48e2bd5688462"} Jul 20 17:58:22.956209 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:22.955642 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" podStartSLOduration=1.955622314 podStartE2EDuration="1.955622314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:22.95441787 +0000 UTC m=+132.082535037" watchObservedRunningTime="2026-07-20 17:58:22.955622314 +0000 UTC m=+132.083739482" Jul 20 17:58:24.940543 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:24.940508 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jlzvz" event={"ID":"77543f6c-716c-4da5-871f-dca1a0db25b8","Type":"ContainerStarted","Data":"436ddca762f0134d482c6d84eea694593d74a769ba1e22a4abd13099ed17c2e6"} Jul 20 17:58:27.956580 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.956533 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" event={"ID":"9598a58c-3f0b-4c79-8f0c-958df137ef19","Type":"ContainerStarted","Data":"9eaa4ea64869544c9364161616d650787c01f4946059b1cff64ed2483e22c1ab"} Jul 20 17:58:27.958128 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.958100 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" event={"ID":"3c94574e-4dc8-4b76-b552-2bc13e9d25c7","Type":"ContainerStarted","Data":"252f258c4f2baee3ad3af7499d2ccb7a4b748ddeee2242eea90afc9e631c19e2"} Jul 20 17:58:27.958303 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.958288 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:27.959857 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.959831 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ts7vs" event={"ID":"ed285dc0-4058-4afa-a620-42799e4245ed","Type":"ContainerStarted","Data":"e0641bf96ad8957e430b6e1d55d2ed027e1de6518504a9f593c10ff14a4f011b"} Jul 20 17:58:27.961701 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.961678 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jlzvz" event={"ID":"77543f6c-716c-4da5-871f-dca1a0db25b8","Type":"ContainerStarted","Data":"4b5c04184117d65b7414c0202be69941708aed0bb75b77688ae0b0dc857ca9d7"} Jul 20 17:58:27.963418 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.963402 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" Jul 20 17:58:27.974092 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.974043 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6479546b4b-2fzxh" podStartSLOduration=2.321447227 podStartE2EDuration="6.974026158s" totalImagesPullingTime="4.652578931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:27.97291221 +0000 UTC m=+137.101029377" watchObservedRunningTime="2026-07-20 17:58:27.974026158 +0000 UTC m=+137.102143320" Jul 20 17:58:27.988794 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:27.988750 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hwmq4" podStartSLOduration=2.336599016 podStartE2EDuration="6.988734236s" totalImagesPullingTime="4.65213522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:27.988333176 +0000 UTC m=+137.116450343" watchObservedRunningTime="2026-07-20 17:58:27.988734236 +0000 UTC m=+137.116851405" Jul 20 17:58:28.006820 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:28.006767 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jlzvz" podStartSLOduration=134.652702586 podStartE2EDuration="2m16.006751579s" totalImagesPullingTime="1.354048993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:28.006078196 +0000 UTC m=+137.134195351" watchObservedRunningTime="2026-07-20 17:58:28.006751579 +0000 UTC m=+137.134868744" Jul 20 17:58:31.481720 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:31.481685 2567 patch_prober.go:28] interesting pod/image-registry-75d765bb8c-h6vjr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 17:58:31.482200 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:31.481738 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:58:31.981991 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:31.981954 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ts7vs" event={"ID":"ed285dc0-4058-4afa-a620-42799e4245ed","Type":"ContainerStarted","Data":"4de22cbf5ee5551209a357f8b728fa4fb0ebf4d945699609285c10474785a1cc"} Jul 20 17:58:32.006817 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:32.006764 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ts7vs" podStartSLOduration=33.058926409 podStartE2EDuration="47.006747063s" totalImagesPullingTime="13.947820654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:57:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:32.006053497 +0000 UTC m=+141.134170687" watchObservedRunningTime="2026-07-20 17:58:32.006747063 +0000 UTC m=+141.134864229" Jul 20 17:58:36.687279 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.687240 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h4jqt"] Jul 20 17:58:36.690441 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.690422 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.693288 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.693268 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 17:58:36.693288 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.693282 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 17:58:36.693496 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.693356 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 17:58:36.693496 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.693422 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 17:58:36.694195 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.694168 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 17:58:36.694268 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.694176 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 17:58:36.695151 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.695137 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bqxsm\"" Jul 20 17:58:36.806399 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806360 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806399 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806403 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26m2x\" (UniqueName: \"kubernetes.io/projected/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-kube-api-access-26m2x\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806643 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-metrics-client-ca\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806643 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806505 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-tls\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806643 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806570 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-wtmp\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806643 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806666 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-sys\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-textfile\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.806830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.806718 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-root\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907103 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-sys\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907103 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907106 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-textfile\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907124 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-root\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907199 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-root\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907211 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-sys\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907244 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907271 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26m2x\" (UniqueName: \"kubernetes.io/projected/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-kube-api-access-26m2x\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907308 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-metrics-client-ca\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-tls\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907362 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907357 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-wtmp\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907699 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907699 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907469 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-textfile\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907699 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:36.907489 2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 17:58:36.907699 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907545 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-wtmp\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.907699 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:36.907574 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-tls podName:2bfa4f73-8012-4ef9-8e5d-f99e067a6895 nodeName:}" failed. No retries permitted until 2026-07-20 17:58:37.407550546 +0000 UTC m=+146.535667705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-tls") pod "node-exporter-h4jqt" (UID: "2bfa4f73-8012-4ef9-8e5d-f99e067a6895") : secret "node-exporter-tls" not found Jul 20 17:58:36.907971 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907950 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.908011 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.907950 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-metrics-client-ca\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.909523 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.909505 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:36.920921 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:36.920900 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26m2x\" (UniqueName: \"kubernetes.io/projected/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-kube-api-access-26m2x\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:37.410839 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:37.410788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-tls\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:37.413017 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:37.412999 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfa4f73-8012-4ef9-8e5d-f99e067a6895-node-exporter-tls\") pod \"node-exporter-h4jqt\" (UID: \"2bfa4f73-8012-4ef9-8e5d-f99e067a6895\") " pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:37.599118 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:37.599072 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h4jqt" Jul 20 17:58:37.611785 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:37.611759 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfa4f73_8012_4ef9_8e5d_f99e067a6895.slice/crio-fbed7e2fa66c031fc7bc77eead3c12d53ed9d79305201159fd4e15c16065a49c WatchSource:0}: Error finding container fbed7e2fa66c031fc7bc77eead3c12d53ed9d79305201159fd4e15c16065a49c: Status 404 returned error can't find the container with id fbed7e2fa66c031fc7bc77eead3c12d53ed9d79305201159fd4e15c16065a49c Jul 20 17:58:38.002784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:38.002748 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4jqt" event={"ID":"2bfa4f73-8012-4ef9-8e5d-f99e067a6895","Type":"ContainerStarted","Data":"fbed7e2fa66c031fc7bc77eead3c12d53ed9d79305201159fd4e15c16065a49c"} Jul 20 17:58:39.008464 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.008378 2567 generic.go:356] "Generic (PLEG): container finished" podID="2bfa4f73-8012-4ef9-8e5d-f99e067a6895" containerID="27540bb22a5266ba8f7e6dc6f9270d6a3829a933872bdae5540d7d3bcfe4191a" exitCode=0 Jul 20 17:58:39.008806 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.008466 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4jqt" event={"ID":"2bfa4f73-8012-4ef9-8e5d-f99e067a6895","Type":"ContainerDied","Data":"27540bb22a5266ba8f7e6dc6f9270d6a3829a933872bdae5540d7d3bcfe4191a"} Jul 20 17:58:39.708105 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.708064 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-76f9875999-rrn98"] Jul 20 17:58:39.711789 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.711766 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.715723 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.715686 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 20 17:58:39.715723 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.715712 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j8gj6\"" Jul 20 17:58:39.716624 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.716601 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 20 17:58:39.716727 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.716646 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 20 17:58:39.716727 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.716658 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-813h9cqjm5o9o\"" Jul 20 17:58:39.716932 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.716916 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 20 17:58:39.718666 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.718644 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 20 17:58:39.733445 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.733418 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-76f9875999-rrn98"] Jul 20 17:58:39.830499 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/043abbe2-ad48-4653-8c10-677fc1c69109-metrics-client-ca\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830499 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830502 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z727\" (UniqueName: \"kubernetes.io/projected/043abbe2-ad48-4653-8c10-677fc1c69109-kube-api-access-6z727\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830695 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830525 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830695 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830547 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-grpc-tls\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830695 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830695 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830683 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830842 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-tls\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.830842 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.830777 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.931938 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.931902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-tls\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.931938 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.931938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932210 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.931962 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/043abbe2-ad48-4653-8c10-677fc1c69109-metrics-client-ca\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932210 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.931979 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z727\" (UniqueName: \"kubernetes.io/projected/043abbe2-ad48-4653-8c10-677fc1c69109-kube-api-access-6z727\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932210 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.932026 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932210 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.932065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-grpc-tls\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932210 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.932093 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932210 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.932127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.932899 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.932873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/043abbe2-ad48-4653-8c10-677fc1c69109-metrics-client-ca\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.934693 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.934670 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.934810 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.934693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-tls\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.935001 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.934980 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-grpc-tls\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.935075 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.935036 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.935075 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.935054 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.935499 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.935478 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/043abbe2-ad48-4653-8c10-677fc1c69109-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:39.941704 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:39.941682 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z727\" (UniqueName: \"kubernetes.io/projected/043abbe2-ad48-4653-8c10-677fc1c69109-kube-api-access-6z727\") pod \"thanos-querier-76f9875999-rrn98\" (UID: \"043abbe2-ad48-4653-8c10-677fc1c69109\") " pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:40.015624 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:40.015539 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4jqt" event={"ID":"2bfa4f73-8012-4ef9-8e5d-f99e067a6895","Type":"ContainerStarted","Data":"7a5e4eb12fd0ef4d6cc02a58d167e981b2cd830faf3a9e1f540cf1a02f0e2433"} Jul 20 17:58:40.015624 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:40.015609 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4jqt" event={"ID":"2bfa4f73-8012-4ef9-8e5d-f99e067a6895","Type":"ContainerStarted","Data":"27ecd31f58621a401b7814ec7e44a1a285aa5300f3e265b81bda0de62f918101"} Jul 20 17:58:40.022049 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:40.022025 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:40.040050 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:40.040002 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h4jqt" podStartSLOduration=2.9485233109999998 podStartE2EDuration="4.03998584s" totalImagesPullingTime="1.021557126s" totalInitContainerRuntime="69.905403ms" isStatefulPod=true podCreationTimestamp="2026-07-20 17:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:40.036603361 +0000 UTC m=+149.164720527" watchObservedRunningTime="2026-07-20 17:58:40.03998584 +0000 UTC m=+149.168103015" Jul 20 17:58:40.177998 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:40.177967 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-76f9875999-rrn98"] Jul 20 17:58:40.181225 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:40.181173 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043abbe2_ad48_4653_8c10_677fc1c69109.slice/crio-c270721e80f05c33eb4b4358526758510aa41b3d44419cde9afeeb389f16de39 WatchSource:0}: Error finding container c270721e80f05c33eb4b4358526758510aa41b3d44419cde9afeeb389f16de39: Status 404 returned error can't find the container with id c270721e80f05c33eb4b4358526758510aa41b3d44419cde9afeeb389f16de39 Jul 20 17:58:41.019788 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.019755 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"c270721e80f05c33eb4b4358526758510aa41b3d44419cde9afeeb389f16de39"} Jul 20 17:58:41.420579 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.420538 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb"] Jul 20 17:58:41.424036 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.424007 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:41.426893 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.426867 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qkjml\"" Jul 20 17:58:41.427295 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.427274 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 20 17:58:41.439494 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.439469 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67d4c848d8-8kv2n"] Jul 20 17:58:41.442844 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.442822 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.446112 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.446090 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5lzmx\"" Jul 20 17:58:41.447014 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.446820 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 17:58:41.447014 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.446835 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 17:58:41.447014 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.446863 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 17:58:41.447014 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.446893 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 17:58:41.447360 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.447340 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 17:58:41.447430 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.447349 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 17:58:41.447863 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.447840 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 17:58:41.455979 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.455546 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 17:58:41.458014 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.457989 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d4c848d8-8kv2n"] Jul 20 17:58:41.481894 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.481867 2567 patch_prober.go:28] interesting pod/image-registry-75d765bb8c-h6vjr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 17:58:41.482026 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.481922 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:58:41.491379 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.491354 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb"] Jul 20 17:58:41.544600 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544568 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-oauth-serving-cert\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.544600 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544603 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-serving-cert\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.544830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544625 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2pw2\" (UniqueName: \"kubernetes.io/projected/67719987-9554-49b7-ad4c-b68f29abfd2f-kube-api-access-d2pw2\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:41.544830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544654 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-oauth-config\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.544830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkw2g\" (UniqueName: \"kubernetes.io/projected/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-kube-api-access-dkw2g\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.544830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544770 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67719987-9554-49b7-ad4c-b68f29abfd2f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:41.544830 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-trusted-ca-bundle\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.545002 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544846 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-config\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.545002 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.544876 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-service-ca\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646191 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-oauth-serving-cert\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646380 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-serving-cert\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646380 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646230 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2pw2\" (UniqueName: \"kubernetes.io/projected/67719987-9554-49b7-ad4c-b68f29abfd2f-kube-api-access-d2pw2\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:41.646380 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646255 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-oauth-config\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646380 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkw2g\" (UniqueName: \"kubernetes.io/projected/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-kube-api-access-dkw2g\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646380 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646330 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67719987-9554-49b7-ad4c-b68f29abfd2f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:41.646380 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646348 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-trusted-ca-bundle\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646755 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646386 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-config\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646755 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646434 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-service-ca\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.646755 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:41.646499 2567 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 20 17:58:41.646755 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:41.646595 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67719987-9554-49b7-ad4c-b68f29abfd2f-monitoring-plugin-cert podName:67719987-9554-49b7-ad4c-b68f29abfd2f nodeName:}" failed. No retries permitted until 2026-07-20 17:58:42.146553571 +0000 UTC m=+151.274670721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/67719987-9554-49b7-ad4c-b68f29abfd2f-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-7lqxb" (UID: "67719987-9554-49b7-ad4c-b68f29abfd2f") : secret "monitoring-plugin-cert" not found Jul 20 17:58:41.647002 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.646976 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-oauth-serving-cert\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.647076 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.647060 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-service-ca\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.647372 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.647348 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-config\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.647445 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.647393 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-trusted-ca-bundle\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.648813 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.648794 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-oauth-config\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.648901 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.648852 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-serving-cert\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.660089 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.660063 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2pw2\" (UniqueName: \"kubernetes.io/projected/67719987-9554-49b7-ad4c-b68f29abfd2f-kube-api-access-d2pw2\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:41.661274 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.661255 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkw2g\" (UniqueName: \"kubernetes.io/projected/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-kube-api-access-dkw2g\") pod \"console-67d4c848d8-8kv2n\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.756725 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.756643 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:41.907749 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.907723 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d4c848d8-8kv2n"] Jul 20 17:58:41.910416 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:41.910387 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a50acc1_c6f8_4b6c_8391_27c6e72acda0.slice/crio-22c1ce3076b9ef7534e9c8add54acd9838cde73785c7708e3450b831e0280179 WatchSource:0}: Error finding container 22c1ce3076b9ef7534e9c8add54acd9838cde73785c7708e3450b831e0280179: Status 404 returned error can't find the container with id 22c1ce3076b9ef7534e9c8add54acd9838cde73785c7708e3450b831e0280179 Jul 20 17:58:41.932908 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.932874 2567 patch_prober.go:28] interesting pod/image-registry-6989456fd7-mfs4w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 17:58:41.933036 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:41.932935 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" podUID="c393437a-06b7-4663-a90f-ef4417c866d1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:58:42.028826 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:42.028733 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4c848d8-8kv2n" event={"ID":"4a50acc1-c6f8-4b6c-8391-27c6e72acda0","Type":"ContainerStarted","Data":"22c1ce3076b9ef7534e9c8add54acd9838cde73785c7708e3450b831e0280179"} Jul 20 17:58:42.151291 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:42.151240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67719987-9554-49b7-ad4c-b68f29abfd2f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:42.153686 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:42.153663 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67719987-9554-49b7-ad4c-b68f29abfd2f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-7lqxb\" (UID: \"67719987-9554-49b7-ad4c-b68f29abfd2f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:42.337040 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:42.337000 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:42.781365 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:42.781339 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb"] Jul 20 17:58:42.789562 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:42.789531 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67719987_9554_49b7_ad4c_b68f29abfd2f.slice/crio-6c2a282e3f47b95ffa4067eb6bad7e63c8b29807ed3d485ff16370386c8cb609 WatchSource:0}: Error finding container 6c2a282e3f47b95ffa4067eb6bad7e63c8b29807ed3d485ff16370386c8cb609: Status 404 returned error can't find the container with id 6c2a282e3f47b95ffa4067eb6bad7e63c8b29807ed3d485ff16370386c8cb609 Jul 20 17:58:43.035715 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:43.035674 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"9338ecb0886be5f2f6d9bb4dfe44df519179d40a275a590da614853167d4160d"} Jul 20 17:58:43.036174 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:43.035771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"a513506f89206110f20088e942ad3f3120879460a9dc95058d5ab2ee6b43d3c0"} Jul 20 17:58:43.036174 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:43.035813 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"5b28578342fc7514b68c6a3ec5d73b77841cbf70d0b010dce8dea2d0fd068419"} Jul 20 17:58:43.036935 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:43.036898 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" event={"ID":"67719987-9554-49b7-ad4c-b68f29abfd2f","Type":"ContainerStarted","Data":"6c2a282e3f47b95ffa4067eb6bad7e63c8b29807ed3d485ff16370386c8cb609"} Jul 20 17:58:43.941064 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:43.941013 2567 patch_prober.go:28] interesting pod/image-registry-6989456fd7-mfs4w container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 17:58:43.941259 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:43.941082 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" podUID="c393437a-06b7-4663-a90f-ef4417c866d1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:58:46.053223 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.053167 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"5d1bdcaca1e8aafaafd618f3113fc423f4924c296058a677711eeb0c37e54433"} Jul 20 17:58:46.053710 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.053270 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"9e699e078ff491f66a0c49a624026373eebb4c08b42922392348171df760e527"} Jul 20 17:58:46.053710 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.053317 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" event={"ID":"043abbe2-ad48-4653-8c10-677fc1c69109","Type":"ContainerStarted","Data":"8aaa9b51361876aedbbf8d56547f6123be1b9e4c26a7da6221644575e3876d7d"} Jul 20 17:58:46.053710 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.053377 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:46.054928 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.054904 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4c848d8-8kv2n" event={"ID":"4a50acc1-c6f8-4b6c-8391-27c6e72acda0","Type":"ContainerStarted","Data":"5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a"} Jul 20 17:58:46.056406 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.056376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" event={"ID":"67719987-9554-49b7-ad4c-b68f29abfd2f","Type":"ContainerStarted","Data":"18631000fad6eabde2896722b359345ddcb856f28d516b7335318f44ae925740"} Jul 20 17:58:46.056553 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.056530 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:46.061555 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.061534 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" Jul 20 17:58:46.078062 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.078018 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" podStartSLOduration=1.917477659 podStartE2EDuration="7.078003133s" totalImagesPullingTime="5.160525474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:58:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:46.076767913 +0000 UTC m=+155.204885097" watchObservedRunningTime="2026-07-20 17:58:46.078003133 +0000 UTC m=+155.206120299" Jul 20 17:58:46.098609 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.098565 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-7lqxb" podStartSLOduration=2.343600299 podStartE2EDuration="5.098551528s" totalImagesPullingTime="2.754951229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:46.097492458 +0000 UTC m=+155.225609624" watchObservedRunningTime="2026-07-20 17:58:46.098551528 +0000 UTC m=+155.226668694" Jul 20 17:58:46.115376 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.115324 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67d4c848d8-8kv2n" podStartSLOduration=1.480705864 podStartE2EDuration="5.115310144s" totalImagesPullingTime="3.63460428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:46.113507814 +0000 UTC m=+155.241625008" watchObservedRunningTime="2026-07-20 17:58:46.115310144 +0000 UTC m=+155.243427310" Jul 20 17:58:46.499822 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.499724 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" containerID="cri-o://f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1" gracePeriod=30 Jul 20 17:58:46.737334 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.737311 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:58:46.897437 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897374 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-certificates\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897467 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-bound-sa-token\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897507 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-trusted-ca\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897538 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a848c84a-cd5e-42bf-9f14-c367f9a9f627-ca-trust-extracted\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897568 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-installation-pull-secrets\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897595 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj77j\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-kube-api-access-wj77j\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897651 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897632 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-image-registry-private-configuration\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897929 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897665 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") pod \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\" (UID: \"a848c84a-cd5e-42bf-9f14-c367f9a9f627\") " Jul 20 17:58:46.897997 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.897965 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:58:46.898064 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.898035 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:58:46.900068 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.899995 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:58:46.900068 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.900058 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 17:58:46.900291 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.900127 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-kube-api-access-wj77j" (OuterVolumeSpecName: "kube-api-access-wj77j") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "kube-api-access-wj77j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:58:46.900384 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.900356 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:58:46.900434 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.900364 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 17:58:46.909234 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.909206 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a848c84a-cd5e-42bf-9f14-c367f9a9f627-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a848c84a-cd5e-42bf-9f14-c367f9a9f627" (UID: "a848c84a-cd5e-42bf-9f14-c367f9a9f627"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:58:46.999258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999215 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj77j\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-kube-api-access-wj77j\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999248 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-image-registry-private-configuration\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999258 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-tls\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999258 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999267 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-registry-certificates\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999536 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999276 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a848c84a-cd5e-42bf-9f14-c367f9a9f627-bound-sa-token\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999536 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999285 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a848c84a-cd5e-42bf-9f14-c367f9a9f627-trusted-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999536 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999293 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a848c84a-cd5e-42bf-9f14-c367f9a9f627-ca-trust-extracted\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:46.999536 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:46.999301 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a848c84a-cd5e-42bf-9f14-c367f9a9f627-installation-pull-secrets\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 17:58:47.061779 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.061746 2567 generic.go:356] "Generic (PLEG): container finished" podID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerID="f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1" exitCode=0 Jul 20 17:58:47.062263 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.061795 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" Jul 20 17:58:47.062263 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.061839 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" event={"ID":"a848c84a-cd5e-42bf-9f14-c367f9a9f627","Type":"ContainerDied","Data":"f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1"} Jul 20 17:58:47.062263 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.061887 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75d765bb8c-h6vjr" event={"ID":"a848c84a-cd5e-42bf-9f14-c367f9a9f627","Type":"ContainerDied","Data":"b658e974ab631f9e25feb5f1a9a4043c105b32b486334e2ee92192d71d020114"} Jul 20 17:58:47.062263 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.061908 2567 scope.go:117] "RemoveContainer" containerID="f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1" Jul 20 17:58:47.071419 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.071395 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-76f9875999-rrn98" Jul 20 17:58:47.076817 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.076799 2567 scope.go:117] "RemoveContainer" containerID="f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1" Jul 20 17:58:47.077130 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:47.077112 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1\": container with ID starting with f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1 not found: ID does not exist" containerID="f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1" Jul 20 17:58:47.077223 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.077143 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1"} err="failed to get container status \"f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1\": rpc error: code = NotFound desc = could not find container \"f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1\": container with ID starting with f3bf1ec5c77a0f0bedf826f973ab107e77ce580eb21644be3a34fb1b4fda13c1 not found: ID does not exist" Jul 20 17:58:47.118406 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.118377 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75d765bb8c-h6vjr"] Jul 20 17:58:47.122560 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.122532 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75d765bb8c-h6vjr"] Jul 20 17:58:47.438950 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:47.438916 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" path="/var/lib/kubelet/pods/a848c84a-cd5e-42bf-9f14-c367f9a9f627/volumes" Jul 20 17:58:49.209757 ip-10-0-133-135 kubenswrapper[2567]: E0720 17:58:49.209715 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pvwbr" podUID="45dc5f22-b96b-457c-b060-a655c6b8adf0" Jul 20 17:58:50.073781 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:50.073753 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvwbr" Jul 20 17:58:51.757593 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:51.757560 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:51.758030 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:51.757625 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:58:51.758732 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:51.758711 2567 patch_prober.go:28] interesting pod/console-67d4c848d8-8kv2n container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body= Jul 20 17:58:51.758813 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:51.758751 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-67d4c848d8-8kv2n" podUID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" Jul 20 17:58:51.931962 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:51.931929 2567 patch_prober.go:28] interesting pod/image-registry-6989456fd7-mfs4w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 17:58:51.932107 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:51.931990 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" podUID="c393437a-06b7-4663-a90f-ef4417c866d1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:58:53.939361 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:53.939333 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6989456fd7-mfs4w" Jul 20 17:58:54.162908 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:54.162857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:58:54.165173 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:54.165147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45dc5f22-b96b-457c-b060-a655c6b8adf0-metrics-tls\") pod \"dns-default-pvwbr\" (UID: \"45dc5f22-b96b-457c-b060-a655c6b8adf0\") " pod="openshift-dns/dns-default-pvwbr" Jul 20 17:58:54.277626 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:54.277538 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m6nkv\"" Jul 20 17:58:54.284809 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:54.284786 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvwbr" Jul 20 17:58:54.444933 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:54.444909 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvwbr"] Jul 20 17:58:54.447385 ip-10-0-133-135 kubenswrapper[2567]: W0720 17:58:54.447358 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45dc5f22_b96b_457c_b060_a655c6b8adf0.slice/crio-7a078959620de1e2361a1603fb81445f083fcdb9640d4afb231869aa77a58b39 WatchSource:0}: Error finding container 7a078959620de1e2361a1603fb81445f083fcdb9640d4afb231869aa77a58b39: Status 404 returned error can't find the container with id 7a078959620de1e2361a1603fb81445f083fcdb9640d4afb231869aa77a58b39 Jul 20 17:58:55.091519 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:55.091485 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvwbr" event={"ID":"45dc5f22-b96b-457c-b060-a655c6b8adf0","Type":"ContainerStarted","Data":"7a078959620de1e2361a1603fb81445f083fcdb9640d4afb231869aa77a58b39"} Jul 20 17:58:57.102401 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:57.102364 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvwbr" event={"ID":"45dc5f22-b96b-457c-b060-a655c6b8adf0","Type":"ContainerStarted","Data":"b99c6d33d5fc6da9f0a64874a0fc7bd718edc7d91a34c1143bdac29618a8e801"} Jul 20 17:58:57.102816 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:57.102452 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvwbr" event={"ID":"45dc5f22-b96b-457c-b060-a655c6b8adf0","Type":"ContainerStarted","Data":"d472d0631ae765f92075c4a721ac48ff3d05a4e79e7e93d863bedba36a5b398e"} Jul 20 17:58:58.106459 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:58.106429 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pvwbr" Jul 20 17:58:58.152784 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:58:58.152726 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pvwbr" podStartSLOduration=129.771124402 podStartE2EDuration="2m12.152709947s" totalImagesPullingTime="2.381585545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:58:58.15145976 +0000 UTC m=+167.279576925" watchObservedRunningTime="2026-07-20 17:58:58.152709947 +0000 UTC m=+167.280827110" Jul 20 17:59:01.120391 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:01.120354 2567 generic.go:356] "Generic (PLEG): container finished" podID="5463db92-b0e2-4bd1-b601-c8d9349830a1" containerID="02cb5bd2ec29d984e5d91a803738d0e602d45f2b0a5f6fe34e6e504eaef5b5a7" exitCode=0 Jul 20 17:59:01.120766 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:01.120429 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" event={"ID":"5463db92-b0e2-4bd1-b601-c8d9349830a1","Type":"ContainerDied","Data":"02cb5bd2ec29d984e5d91a803738d0e602d45f2b0a5f6fe34e6e504eaef5b5a7"} Jul 20 17:59:01.120811 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:01.120786 2567 scope.go:117] "RemoveContainer" containerID="02cb5bd2ec29d984e5d91a803738d0e602d45f2b0a5f6fe34e6e504eaef5b5a7" Jul 20 17:59:01.761976 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:01.761941 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:59:01.765846 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:01.765823 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 17:59:02.127716 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:02.127685 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lvvhx" event={"ID":"5463db92-b0e2-4bd1-b601-c8d9349830a1","Type":"ContainerStarted","Data":"03d0fc10c9552ca21e43ee796d66e2d678984393b01381081e96545d35f30382"} Jul 20 17:59:05.140555 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:05.140521 2567 generic.go:356] "Generic (PLEG): container finished" podID="d275ccd4-1373-4b29-bb57-ca14e91580d9" containerID="54d9e600218f4551d2bbe878b967a1d21cdb182c955ef37c1a7a902a75b13dd6" exitCode=0 Jul 20 17:59:05.141026 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:05.140584 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" event={"ID":"d275ccd4-1373-4b29-bb57-ca14e91580d9","Type":"ContainerDied","Data":"54d9e600218f4551d2bbe878b967a1d21cdb182c955ef37c1a7a902a75b13dd6"} Jul 20 17:59:05.141026 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:05.140876 2567 scope.go:117] "RemoveContainer" containerID="54d9e600218f4551d2bbe878b967a1d21cdb182c955ef37c1a7a902a75b13dd6" Jul 20 17:59:06.073666 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:06.073617 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvwbr_45dc5f22-b96b-457c-b060-a655c6b8adf0/dns/0.log" Jul 20 17:59:06.080579 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:06.080549 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvwbr_45dc5f22-b96b-457c-b060-a655c6b8adf0/kube-rbac-proxy/0.log" Jul 20 17:59:06.146432 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:06.146399 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-nkrsq" event={"ID":"d275ccd4-1373-4b29-bb57-ca14e91580d9","Type":"ContainerStarted","Data":"eab17c04026c633ded28ec5dedcace3d267e2403e9cce27c45105924d9078327"} Jul 20 17:59:06.784473 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:06.784441 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vjrf4_02060614-726f-4116-8956-25ad464aa2c6/dns-node-resolver/0.log" Jul 20 17:59:08.113219 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:08.113170 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pvwbr" Jul 20 17:59:19.200294 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:19.200245 2567 generic.go:356] "Generic (PLEG): container finished" podID="8c8d84a0-d438-4540-91a6-7c7d7bd04b2d" containerID="32ebcac1c9f129e47a9950117958cd4e106d8fb898efd225a38cfac4d20b9903" exitCode=0 Jul 20 17:59:19.200748 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:19.200313 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" event={"ID":"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d","Type":"ContainerDied","Data":"32ebcac1c9f129e47a9950117958cd4e106d8fb898efd225a38cfac4d20b9903"} Jul 20 17:59:19.200748 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:19.200745 2567 scope.go:117] "RemoveContainer" containerID="32ebcac1c9f129e47a9950117958cd4e106d8fb898efd225a38cfac4d20b9903" Jul 20 17:59:20.206304 ip-10-0-133-135 kubenswrapper[2567]: I0720 17:59:20.206269 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7blsw" event={"ID":"8c8d84a0-d438-4540-91a6-7c7d7bd04b2d","Type":"ContainerStarted","Data":"9de99638cc09720f1cf6fbae7816ee3eb04cde4a17c6b0a0d1433f222c75f488"} Jul 20 18:00:00.973291 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.973249 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69d99cd586-tmfs6"] Jul 20 18:00:00.973867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.973671 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" Jul 20 18:00:00.973867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.973689 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" Jul 20 18:00:00.973867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.973776 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a848c84a-cd5e-42bf-9f14-c367f9a9f627" containerName="registry" Jul 20 18:00:00.977916 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.977893 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:00.987167 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.987147 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 20 18:00:00.987325 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.987307 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 20 18:00:00.987375 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.987330 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 20 18:00:00.987854 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.987837 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 20 18:00:00.988298 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.988283 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 20 18:00:00.997450 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.997420 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8xhs7\"" Jul 20 18:00:00.999332 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:00.999312 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69d99cd586-tmfs6"] Jul 20 18:00:01.004147 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.004127 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 20 18:00:01.020916 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.020893 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-telemeter-client-tls\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021047 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.020938 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-metrics-client-ca\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021047 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.020995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-serving-certs-ca-bundle\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021047 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.021041 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-federate-client-tls\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021231 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.021075 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021231 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.021097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021231 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.021154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v9vq\" (UniqueName: \"kubernetes.io/projected/b9b91dbb-0910-4e5c-95e1-f08a9e593748-kube-api-access-4v9vq\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.021362 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.021230 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-secret-telemeter-client\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122453 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122414 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-secret-telemeter-client\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122453 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-telemeter-client-tls\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-metrics-client-ca\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122513 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-serving-certs-ca-bundle\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-federate-client-tls\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122618 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.122934 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.122736 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v9vq\" (UniqueName: \"kubernetes.io/projected/b9b91dbb-0910-4e5c-95e1-f08a9e593748-kube-api-access-4v9vq\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.123422 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.123395 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-metrics-client-ca\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.123563 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.123428 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-serving-certs-ca-bundle\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.123563 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.123539 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b91dbb-0910-4e5c-95e1-f08a9e593748-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.125174 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.125154 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-telemeter-client-tls\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.125319 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.125302 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.125577 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.125558 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-federate-client-tls\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.125636 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.125565 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b9b91dbb-0910-4e5c-95e1-f08a9e593748-secret-telemeter-client\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.131398 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.131379 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v9vq\" (UniqueName: \"kubernetes.io/projected/b9b91dbb-0910-4e5c-95e1-f08a9e593748-kube-api-access-4v9vq\") pod \"telemeter-client-69d99cd586-tmfs6\" (UID: \"b9b91dbb-0910-4e5c-95e1-f08a9e593748\") " pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.286577 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.286480 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" Jul 20 18:00:01.421432 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:01.421400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69d99cd586-tmfs6"] Jul 20 18:00:01.424818 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:00:01.424782 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b91dbb_0910_4e5c_95e1_f08a9e593748.slice/crio-2943b0c7690481db3f03b43bd59fe19682f5819c62a1d00a84d2e637b81c9609 WatchSource:0}: Error finding container 2943b0c7690481db3f03b43bd59fe19682f5819c62a1d00a84d2e637b81c9609: Status 404 returned error can't find the container with id 2943b0c7690481db3f03b43bd59fe19682f5819c62a1d00a84d2e637b81c9609 Jul 20 18:00:02.364081 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:02.364045 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" event={"ID":"b9b91dbb-0910-4e5c-95e1-f08a9e593748","Type":"ContainerStarted","Data":"2943b0c7690481db3f03b43bd59fe19682f5819c62a1d00a84d2e637b81c9609"} Jul 20 18:00:05.377866 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:05.377831 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" event={"ID":"b9b91dbb-0910-4e5c-95e1-f08a9e593748","Type":"ContainerStarted","Data":"5f79cbac0b1fdf750e6984c625716d4a800cb4c9a70a2d8a61566518563fbe56"} Jul 20 18:00:06.383815 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:06.383776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" event={"ID":"b9b91dbb-0910-4e5c-95e1-f08a9e593748","Type":"ContainerStarted","Data":"4d8c0e0e11386ef3dc74e13bcb12fa0753bc6eb0c96681424502ecf493ddf212"} Jul 20 18:00:06.384227 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:06.383842 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" event={"ID":"b9b91dbb-0910-4e5c-95e1-f08a9e593748","Type":"ContainerStarted","Data":"a95c5f47c6f859b168c8df3f2b6220eaf5ca03df83ffcb09f94393ca27033b93"} Jul 20 18:00:06.407097 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:06.407045 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69d99cd586-tmfs6" podStartSLOduration=1.9697256909999998 podStartE2EDuration="6.407031065s" totalImagesPullingTime="4.437305374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:00:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:00:06.405760195 +0000 UTC m=+235.533877360" watchObservedRunningTime="2026-07-20 18:00:06.407031065 +0000 UTC m=+235.535148230" Jul 20 18:00:07.110530 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.110496 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58c96c7b88-wxzjc"] Jul 20 18:00:07.112736 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.112717 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.125457 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.125431 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58c96c7b88-wxzjc"] Jul 20 18:00:07.171782 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-serving-cert\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.171950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171792 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-oauth-config\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.171950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-trusted-ca-bundle\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.171950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171858 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-oauth-serving-cert\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.171950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171899 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-service-ca\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.171950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171935 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmbvw\" (UniqueName: \"kubernetes.io/projected/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-kube-api-access-nmbvw\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.172146 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.171963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-config\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272479 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-service-ca\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272639 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272492 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmbvw\" (UniqueName: \"kubernetes.io/projected/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-kube-api-access-nmbvw\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272639 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272604 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-config\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272710 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-serving-cert\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272747 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-oauth-config\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272747 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272725 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-trusted-ca-bundle\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.272819 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.272745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-oauth-serving-cert\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.273242 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.273208 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-service-ca\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.273402 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.273381 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-config\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.273479 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.273382 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-oauth-serving-cert\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.273791 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.273771 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-trusted-ca-bundle\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.275154 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.275133 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-oauth-config\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.275266 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.275211 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-serving-cert\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.280785 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.280762 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmbvw\" (UniqueName: \"kubernetes.io/projected/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-kube-api-access-nmbvw\") pod \"console-58c96c7b88-wxzjc\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.422701 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.422600 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:07.552488 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:07.552463 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58c96c7b88-wxzjc"] Jul 20 18:00:07.554712 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:00:07.554684 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67d89a3_adc9_46cf_9ca9_78118fb0d5d9.slice/crio-5cd6f363e4cf47ea9c24fcf9250aa05a3b5dcfe185fac8b46912b25a088f3f8e WatchSource:0}: Error finding container 5cd6f363e4cf47ea9c24fcf9250aa05a3b5dcfe185fac8b46912b25a088f3f8e: Status 404 returned error can't find the container with id 5cd6f363e4cf47ea9c24fcf9250aa05a3b5dcfe185fac8b46912b25a088f3f8e Jul 20 18:00:08.398136 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:08.398094 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c96c7b88-wxzjc" event={"ID":"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9","Type":"ContainerStarted","Data":"e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017"} Jul 20 18:00:08.398364 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:08.398170 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c96c7b88-wxzjc" event={"ID":"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9","Type":"ContainerStarted","Data":"5cd6f363e4cf47ea9c24fcf9250aa05a3b5dcfe185fac8b46912b25a088f3f8e"} Jul 20 18:00:08.416745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:08.416697 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58c96c7b88-wxzjc" podStartSLOduration=1.416682983 podStartE2EDuration="1.416682983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:00:08.415511654 +0000 UTC m=+237.543628819" watchObservedRunningTime="2026-07-20 18:00:08.416682983 +0000 UTC m=+237.544800148" Jul 20 18:00:16.617986 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.617947 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58c96c7b88-wxzjc"] Jul 20 18:00:16.649780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.649753 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-544ddbdb4-clb26"] Jul 20 18:00:16.662992 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.662968 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.666069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.666014 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-544ddbdb4-clb26"] Jul 20 18:00:16.755692 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755654 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-config\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.755692 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755694 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-oauth-serving-cert\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.755913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755775 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-service-ca\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.755913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755835 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-oauth-config\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.755913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755879 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-serving-cert\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.755913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755897 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-trusted-ca-bundle\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.756058 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.755923 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j99rw\" (UniqueName: \"kubernetes.io/projected/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-kube-api-access-j99rw\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857064 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857011 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-config\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857064 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857068 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-oauth-serving-cert\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-service-ca\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857144 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-oauth-config\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-serving-cert\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857229 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-trusted-ca-bundle\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j99rw\" (UniqueName: \"kubernetes.io/projected/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-kube-api-access-j99rw\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857875 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857842 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-config\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857990 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857883 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-oauth-serving-cert\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.857990 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.857923 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-service-ca\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.858069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.858035 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-trusted-ca-bundle\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.860278 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.860258 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-oauth-config\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.860278 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.860269 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-console-serving-cert\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.865613 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.865591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j99rw\" (UniqueName: \"kubernetes.io/projected/3868656f-d6a9-4603-a945-b4dd0cfcaf1c-kube-api-access-j99rw\") pod \"console-544ddbdb4-clb26\" (UID: \"3868656f-d6a9-4603-a945-b4dd0cfcaf1c\") " pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:16.974013 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:16.973928 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:17.124847 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:17.124810 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-544ddbdb4-clb26"] Jul 20 18:00:17.128054 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:00:17.128030 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3868656f_d6a9_4603_a945_b4dd0cfcaf1c.slice/crio-51720dd870fef258ffe83b7fcbe44fb97520215cf6414db45cd886df17dcf704 WatchSource:0}: Error finding container 51720dd870fef258ffe83b7fcbe44fb97520215cf6414db45cd886df17dcf704: Status 404 returned error can't find the container with id 51720dd870fef258ffe83b7fcbe44fb97520215cf6414db45cd886df17dcf704 Jul 20 18:00:17.423819 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:17.423765 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:17.438614 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:17.438577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544ddbdb4-clb26" event={"ID":"3868656f-d6a9-4603-a945-b4dd0cfcaf1c","Type":"ContainerStarted","Data":"9e90d65d96dc6949562330c91e3328b1f0a4fa856ab5e46831f4a2dc11ea4238"} Jul 20 18:00:17.438767 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:17.438642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544ddbdb4-clb26" event={"ID":"3868656f-d6a9-4603-a945-b4dd0cfcaf1c","Type":"ContainerStarted","Data":"51720dd870fef258ffe83b7fcbe44fb97520215cf6414db45cd886df17dcf704"} Jul 20 18:00:17.453550 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:17.453491 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-544ddbdb4-clb26" podStartSLOduration=1.4534767180000001 podStartE2EDuration="1.453476718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:00:17.452230545 +0000 UTC m=+246.580347721" watchObservedRunningTime="2026-07-20 18:00:17.453476718 +0000 UTC m=+246.581593883" Jul 20 18:00:26.974203 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:26.974152 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:26.974654 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:26.974231 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:26.978852 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:26.978831 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:27.474834 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:27.474805 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-544ddbdb4-clb26" Jul 20 18:00:27.518943 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:27.518910 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d4c848d8-8kv2n"] Jul 20 18:00:41.638435 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.638323 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58c96c7b88-wxzjc" podUID="c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" containerName="console" containerID="cri-o://e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017" gracePeriod=15 Jul 20 18:00:41.880635 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.880609 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:41.972866 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.972790 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-trusted-ca-bundle\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.972866 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.972831 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-service-ca\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.973093 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.972871 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-serving-cert\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.973093 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.972906 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-oauth-serving-cert\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.973093 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.972938 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmbvw\" (UniqueName: \"kubernetes.io/projected/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-kube-api-access-nmbvw\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.973093 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973070 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-oauth-config\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.973329 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973139 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-config\") pod \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\" (UID: \"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9\") " Jul 20 18:00:41.973329 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973289 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:41.973434 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973331 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:41.973558 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973505 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-trusted-ca-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:41.973558 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973528 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:41.973689 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973639 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-config" (OuterVolumeSpecName: "console-config") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:41.973829 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.973804 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:41.975662 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.975609 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:00:41.975780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.975684 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:00:41.975780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:41.975716 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-kube-api-access-nmbvw" (OuterVolumeSpecName: "kube-api-access-nmbvw") pod "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" (UID: "c67d89a3-adc9-46cf-9ca9-78118fb0d5d9"). InnerVolumeSpecName "kube-api-access-nmbvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:00:42.074223 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.074164 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-serving-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:42.074223 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.074217 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-oauth-serving-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:42.074223 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.074231 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmbvw\" (UniqueName: \"kubernetes.io/projected/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-kube-api-access-nmbvw\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:42.074456 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.074246 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-oauth-config\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:42.074456 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.074258 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9-console-config\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:42.532264 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.532226 2567 generic.go:356] "Generic (PLEG): container finished" podID="c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" containerID="e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017" exitCode=0 Jul 20 18:00:42.532264 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.532261 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c96c7b88-wxzjc" Jul 20 18:00:42.532529 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.532303 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c96c7b88-wxzjc" event={"ID":"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9","Type":"ContainerDied","Data":"e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017"} Jul 20 18:00:42.532529 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.532361 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c96c7b88-wxzjc" event={"ID":"c67d89a3-adc9-46cf-9ca9-78118fb0d5d9","Type":"ContainerDied","Data":"5cd6f363e4cf47ea9c24fcf9250aa05a3b5dcfe185fac8b46912b25a088f3f8e"} Jul 20 18:00:42.532529 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.532391 2567 scope.go:117] "RemoveContainer" containerID="e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017" Jul 20 18:00:42.541506 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.541489 2567 scope.go:117] "RemoveContainer" containerID="e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017" Jul 20 18:00:42.541733 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:00:42.541714 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017\": container with ID starting with e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017 not found: ID does not exist" containerID="e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017" Jul 20 18:00:42.541779 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.541744 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017"} err="failed to get container status \"e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017\": rpc error: code = NotFound desc = could not find container \"e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017\": container with ID starting with e8ca9683e961705f4ae1d6204e016abbf15757e2491d128a0cc384f4ba7fc017 not found: ID does not exist" Jul 20 18:00:42.556145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.556118 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58c96c7b88-wxzjc"] Jul 20 18:00:42.562207 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:42.562173 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58c96c7b88-wxzjc"] Jul 20 18:00:43.439294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:43.439260 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" path="/var/lib/kubelet/pods/c67d89a3-adc9-46cf-9ca9-78118fb0d5d9/volumes" Jul 20 18:00:52.541739 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.541671 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67d4c848d8-8kv2n" podUID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" containerName="console" containerID="cri-o://5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a" gracePeriod=15 Jul 20 18:00:52.796885 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.796866 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 18:00:52.969485 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969450 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkw2g\" (UniqueName: \"kubernetes.io/projected/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-kube-api-access-dkw2g\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.969655 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969509 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-oauth-config\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.969655 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969562 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-trusted-ca-bundle\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.969655 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969588 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-serving-cert\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.969655 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969616 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-config\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.969655 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969649 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-service-ca\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.969935 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.969678 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-oauth-serving-cert\") pod \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\" (UID: \"4a50acc1-c6f8-4b6c-8391-27c6e72acda0\") " Jul 20 18:00:52.970175 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.970139 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:52.970305 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.970068 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:52.970305 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.970167 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-config" (OuterVolumeSpecName: "console-config") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:52.970305 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.970294 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:00:52.971783 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.971756 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:00:52.971880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.971782 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:00:52.971880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:52.971808 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-kube-api-access-dkw2g" (OuterVolumeSpecName: "kube-api-access-dkw2g") pod "4a50acc1-c6f8-4b6c-8391-27c6e72acda0" (UID: "4a50acc1-c6f8-4b6c-8391-27c6e72acda0"). InnerVolumeSpecName "kube-api-access-dkw2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070918 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-trusted-ca-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070951 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-serving-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070960 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-config\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070970 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070980 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-oauth-serving-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070988 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkw2g\" (UniqueName: \"kubernetes.io/projected/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-kube-api-access-dkw2g\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.071002 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.070997 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a50acc1-c6f8-4b6c-8391-27c6e72acda0-console-oauth-config\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:00:53.579463 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.579430 2567 generic.go:356] "Generic (PLEG): container finished" podID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" containerID="5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a" exitCode=0 Jul 20 18:00:53.579889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.579475 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4c848d8-8kv2n" Jul 20 18:00:53.579889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.579488 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4c848d8-8kv2n" event={"ID":"4a50acc1-c6f8-4b6c-8391-27c6e72acda0","Type":"ContainerDied","Data":"5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a"} Jul 20 18:00:53.579889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.579525 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4c848d8-8kv2n" event={"ID":"4a50acc1-c6f8-4b6c-8391-27c6e72acda0","Type":"ContainerDied","Data":"22c1ce3076b9ef7534e9c8add54acd9838cde73785c7708e3450b831e0280179"} Jul 20 18:00:53.579889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.579544 2567 scope.go:117] "RemoveContainer" containerID="5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a" Jul 20 18:00:53.587722 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.587704 2567 scope.go:117] "RemoveContainer" containerID="5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a" Jul 20 18:00:53.587983 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:00:53.587964 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a\": container with ID starting with 5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a not found: ID does not exist" containerID="5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a" Jul 20 18:00:53.588039 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.587992 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a"} err="failed to get container status \"5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a\": rpc error: code = NotFound desc = could not find container \"5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a\": container with ID starting with 5df4b919b19b970317b3dd3fb81fd6a6cc83033dd3a57105dfaa136cd3c0411a not found: ID does not exist" Jul 20 18:00:53.600343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.600310 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d4c848d8-8kv2n"] Jul 20 18:00:53.602504 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:53.602482 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67d4c848d8-8kv2n"] Jul 20 18:00:55.439455 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:00:55.439423 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" path="/var/lib/kubelet/pods/4a50acc1-c6f8-4b6c-8391-27c6e72acda0/volumes" Jul 20 18:01:11.267851 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:11.267819 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:01:11.271115 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:11.271091 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:01:11.279630 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:11.279607 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 18:01:27.719745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.719709 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx"] Jul 20 18:01:27.721133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.719986 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" containerName="console" Jul 20 18:01:27.721133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.719996 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" containerName="console" Jul 20 18:01:27.721133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.720007 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" containerName="console" Jul 20 18:01:27.721133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.720013 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" containerName="console" Jul 20 18:01:27.721133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.720085 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a50acc1-c6f8-4b6c-8391-27c6e72acda0" containerName="console" Jul 20 18:01:27.721133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.720098 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67d89a3-adc9-46cf-9ca9-78118fb0d5d9" containerName="console" Jul 20 18:01:27.721977 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.721961 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.724487 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.724463 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 18:01:27.725372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.725356 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nmt99\"" Jul 20 18:01:27.725372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.725367 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 18:01:27.733938 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.733915 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx"] Jul 20 18:01:27.815870 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.815830 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.816044 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.815912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffgfp\" (UniqueName: \"kubernetes.io/projected/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-kube-api-access-ffgfp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.816044 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.815942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.917063 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.917025 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.917245 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.917083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffgfp\" (UniqueName: \"kubernetes.io/projected/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-kube-api-access-ffgfp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.917245 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.917113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.917450 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.917428 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.917527 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.917482 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:27.925437 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:27.925416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffgfp\" (UniqueName: \"kubernetes.io/projected/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-kube-api-access-ffgfp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:28.030872 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:28.030798 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:28.161511 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:28.161477 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx"] Jul 20 18:01:28.164013 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:01:28.163984 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff3bb7c_ffee_4b98_8ff5_ff589abdd6d8.slice/crio-7f05e3361caeac8e1d676a8627ad1ecb69af35327dcc93c87df31911893054ec WatchSource:0}: Error finding container 7f05e3361caeac8e1d676a8627ad1ecb69af35327dcc93c87df31911893054ec: Status 404 returned error can't find the container with id 7f05e3361caeac8e1d676a8627ad1ecb69af35327dcc93c87df31911893054ec Jul 20 18:01:28.166038 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:28.166021 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 18:01:28.720529 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:28.720492 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" event={"ID":"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8","Type":"ContainerStarted","Data":"7f05e3361caeac8e1d676a8627ad1ecb69af35327dcc93c87df31911893054ec"} Jul 20 18:01:36.754398 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:36.754361 2567 generic.go:356] "Generic (PLEG): container finished" podID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerID="eae611c3fc682379eb4c85497f8b9e0dc605d8e5456aa8e25507fee1ee839555" exitCode=0 Jul 20 18:01:36.754786 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:36.754448 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" event={"ID":"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8","Type":"ContainerDied","Data":"eae611c3fc682379eb4c85497f8b9e0dc605d8e5456aa8e25507fee1ee839555"} Jul 20 18:01:42.779777 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:42.779740 2567 generic.go:356] "Generic (PLEG): container finished" podID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerID="596f3663152a4f474b1444705a0307b0677bf6f90124c83f73c0db36bfeaed46" exitCode=0 Jul 20 18:01:42.780220 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:42.779796 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" event={"ID":"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8","Type":"ContainerDied","Data":"596f3663152a4f474b1444705a0307b0677bf6f90124c83f73c0db36bfeaed46"} Jul 20 18:01:52.824548 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:52.824509 2567 generic.go:356] "Generic (PLEG): container finished" podID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerID="51acd112013b045281a425422c0680118b7824aee89182ada86e0e7ba6da3165" exitCode=0 Jul 20 18:01:52.824927 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:52.824596 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" event={"ID":"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8","Type":"ContainerDied","Data":"51acd112013b045281a425422c0680118b7824aee89182ada86e0e7ba6da3165"} Jul 20 18:01:53.954670 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:53.954647 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:01:54.040594 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.040561 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffgfp\" (UniqueName: \"kubernetes.io/projected/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-kube-api-access-ffgfp\") pod \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " Jul 20 18:01:54.040754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.040646 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-util\") pod \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " Jul 20 18:01:54.040754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.040677 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-bundle\") pod \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\" (UID: \"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8\") " Jul 20 18:01:54.041378 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.041353 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-bundle" (OuterVolumeSpecName: "bundle") pod "fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" (UID: "fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:01:54.042727 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.042707 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-kube-api-access-ffgfp" (OuterVolumeSpecName: "kube-api-access-ffgfp") pod "fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" (UID: "fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8"). InnerVolumeSpecName "kube-api-access-ffgfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:01:54.044823 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.044795 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-util" (OuterVolumeSpecName: "util") pod "fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" (UID: "fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:01:54.141259 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.141163 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:01:54.141259 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.141210 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffgfp\" (UniqueName: \"kubernetes.io/projected/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-kube-api-access-ffgfp\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:01:54.141259 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.141221 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:01:54.833317 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.833283 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" event={"ID":"fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8","Type":"ContainerDied","Data":"7f05e3361caeac8e1d676a8627ad1ecb69af35327dcc93c87df31911893054ec"} Jul 20 18:01:54.833541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.833340 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f05e3361caeac8e1d676a8627ad1ecb69af35327dcc93c87df31911893054ec" Jul 20 18:01:54.833541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:01:54.833298 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ld8fx" Jul 20 18:02:00.593346 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593311 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9"] Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593595 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="extract" Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593606 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="extract" Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593619 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="pull" Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593624 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="pull" Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593632 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="util" Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593637 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="util" Jul 20 18:02:00.593816 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.593700 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff3bb7c-ffee-4b98-8ff5-ff589abdd6d8" containerName="extract" Jul 20 18:02:00.595895 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.595880 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.601387 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.601367 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-96677\"" Jul 20 18:02:00.601694 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.601677 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 20 18:02:00.602254 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.602063 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 20 18:02:00.626253 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.626226 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9"] Jul 20 18:02:00.696349 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.696318 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dnbt\" (UniqueName: \"kubernetes.io/projected/b64d82b3-1e51-45ae-b031-767fae7ecdf4-kube-api-access-2dnbt\") pod \"cert-manager-operator-controller-manager-55c444bf86-hnns9\" (UID: \"b64d82b3-1e51-45ae-b031-767fae7ecdf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.696507 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.696363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b64d82b3-1e51-45ae-b031-767fae7ecdf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hnns9\" (UID: \"b64d82b3-1e51-45ae-b031-767fae7ecdf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.796919 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.796877 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dnbt\" (UniqueName: \"kubernetes.io/projected/b64d82b3-1e51-45ae-b031-767fae7ecdf4-kube-api-access-2dnbt\") pod \"cert-manager-operator-controller-manager-55c444bf86-hnns9\" (UID: \"b64d82b3-1e51-45ae-b031-767fae7ecdf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.797075 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.796931 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b64d82b3-1e51-45ae-b031-767fae7ecdf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hnns9\" (UID: \"b64d82b3-1e51-45ae-b031-767fae7ecdf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.797294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.797279 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b64d82b3-1e51-45ae-b031-767fae7ecdf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hnns9\" (UID: \"b64d82b3-1e51-45ae-b031-767fae7ecdf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.806076 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.806056 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dnbt\" (UniqueName: \"kubernetes.io/projected/b64d82b3-1e51-45ae-b031-767fae7ecdf4-kube-api-access-2dnbt\") pod \"cert-manager-operator-controller-manager-55c444bf86-hnns9\" (UID: \"b64d82b3-1e51-45ae-b031-767fae7ecdf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:00.904784 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:00.904694 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" Jul 20 18:02:01.036606 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:01.036427 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9"] Jul 20 18:02:01.039663 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:01.039631 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64d82b3_1e51_45ae_b031_767fae7ecdf4.slice/crio-fc9e01e58555633145b07eb752ec9067daf10d7fb5f73ba1aa0f2b0d47e36f7a WatchSource:0}: Error finding container fc9e01e58555633145b07eb752ec9067daf10d7fb5f73ba1aa0f2b0d47e36f7a: Status 404 returned error can't find the container with id fc9e01e58555633145b07eb752ec9067daf10d7fb5f73ba1aa0f2b0d47e36f7a Jul 20 18:02:01.861404 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:01.861364 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" event={"ID":"b64d82b3-1e51-45ae-b031-767fae7ecdf4","Type":"ContainerStarted","Data":"fc9e01e58555633145b07eb752ec9067daf10d7fb5f73ba1aa0f2b0d47e36f7a"} Jul 20 18:02:02.869477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:02.869445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" event={"ID":"b64d82b3-1e51-45ae-b031-767fae7ecdf4","Type":"ContainerStarted","Data":"657bde54ae7cc56fe1705f782a2ea286964b1e391b92cfecfaa25405f0814f6c"} Jul 20 18:02:02.896818 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:02.896764 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hnns9" podStartSLOduration=1.491727965 podStartE2EDuration="2.896747184s" totalImagesPullingTime="1.405019219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:02:02.894337591 +0000 UTC m=+352.022454753" watchObservedRunningTime="2026-07-20 18:02:02.896747184 +0000 UTC m=+352.024864340" Jul 20 18:02:04.168413 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.168378 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm"] Jul 20 18:02:04.170884 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.170867 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.173501 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.173479 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 18:02:04.174382 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.174367 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 18:02:04.174439 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.174368 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nmt99\"" Jul 20 18:02:04.183261 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.183237 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm"] Jul 20 18:02:04.328438 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.328392 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.328632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.328443 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.328632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.328544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtfj\" (UniqueName: \"kubernetes.io/projected/05df6dcd-1898-4a72-9d7c-10371ad668cf-kube-api-access-gbtfj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.430017 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.429911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.430017 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.429963 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.430300 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.430044 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtfj\" (UniqueName: \"kubernetes.io/projected/05df6dcd-1898-4a72-9d7c-10371ad668cf-kube-api-access-gbtfj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.430416 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.430388 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.430464 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.430397 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.438924 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.438897 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtfj\" (UniqueName: \"kubernetes.io/projected/05df6dcd-1898-4a72-9d7c-10371ad668cf-kube-api-access-gbtfj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.479456 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.479420 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:04.614705 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.614682 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm"] Jul 20 18:02:04.617034 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:04.617008 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05df6dcd_1898_4a72_9d7c_10371ad668cf.slice/crio-0bb48a6b4afd013c3ead57970734c1511aaf6e6d4a0866d09ccaafbe856357fb WatchSource:0}: Error finding container 0bb48a6b4afd013c3ead57970734c1511aaf6e6d4a0866d09ccaafbe856357fb: Status 404 returned error can't find the container with id 0bb48a6b4afd013c3ead57970734c1511aaf6e6d4a0866d09ccaafbe856357fb Jul 20 18:02:04.878988 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.878943 2567 generic.go:356] "Generic (PLEG): container finished" podID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerID="2ba295d2824aabca8083dfdb70eb8b7773ce4eb2dba3974926717e54201666af" exitCode=0 Jul 20 18:02:04.879161 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.879032 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" event={"ID":"05df6dcd-1898-4a72-9d7c-10371ad668cf","Type":"ContainerDied","Data":"2ba295d2824aabca8083dfdb70eb8b7773ce4eb2dba3974926717e54201666af"} Jul 20 18:02:04.879161 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:04.879080 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" event={"ID":"05df6dcd-1898-4a72-9d7c-10371ad668cf","Type":"ContainerStarted","Data":"0bb48a6b4afd013c3ead57970734c1511aaf6e6d4a0866d09ccaafbe856357fb"} Jul 20 18:02:05.786365 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.786275 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n56ls"] Jul 20 18:02:05.788537 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.788514 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:05.791224 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.791059 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 18:02:05.791224 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.791107 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 18:02:05.792010 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.791990 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xj9lp\"" Jul 20 18:02:05.801467 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.801442 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n56ls"] Jul 20 18:02:05.944059 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.944027 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aaedb57-8008-4075-996f-4de38cfb474c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n56ls\" (UID: \"0aaedb57-8008-4075-996f-4de38cfb474c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:05.944059 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:05.944065 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4gs9\" (UniqueName: \"kubernetes.io/projected/0aaedb57-8008-4075-996f-4de38cfb474c-kube-api-access-v4gs9\") pod \"cert-manager-cainjector-559cdd8c87-n56ls\" (UID: \"0aaedb57-8008-4075-996f-4de38cfb474c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:06.044868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.044775 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aaedb57-8008-4075-996f-4de38cfb474c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n56ls\" (UID: \"0aaedb57-8008-4075-996f-4de38cfb474c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:06.044868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.044826 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4gs9\" (UniqueName: \"kubernetes.io/projected/0aaedb57-8008-4075-996f-4de38cfb474c-kube-api-access-v4gs9\") pod \"cert-manager-cainjector-559cdd8c87-n56ls\" (UID: \"0aaedb57-8008-4075-996f-4de38cfb474c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:06.053999 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.053970 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aaedb57-8008-4075-996f-4de38cfb474c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n56ls\" (UID: \"0aaedb57-8008-4075-996f-4de38cfb474c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:06.054137 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.054118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4gs9\" (UniqueName: \"kubernetes.io/projected/0aaedb57-8008-4075-996f-4de38cfb474c-kube-api-access-v4gs9\") pod \"cert-manager-cainjector-559cdd8c87-n56ls\" (UID: \"0aaedb57-8008-4075-996f-4de38cfb474c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:06.101104 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.101067 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" Jul 20 18:02:06.243145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.243104 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n56ls"] Jul 20 18:02:06.843539 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:06.843505 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aaedb57_8008_4075_996f_4de38cfb474c.slice/crio-58e5f4557b5531a9db212cc480991cfc04ae5e61a2b3a8a01650fbb7dc2b88a7 WatchSource:0}: Error finding container 58e5f4557b5531a9db212cc480991cfc04ae5e61a2b3a8a01650fbb7dc2b88a7: Status 404 returned error can't find the container with id 58e5f4557b5531a9db212cc480991cfc04ae5e61a2b3a8a01650fbb7dc2b88a7 Jul 20 18:02:06.889248 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:06.889207 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" event={"ID":"0aaedb57-8008-4075-996f-4de38cfb474c","Type":"ContainerStarted","Data":"58e5f4557b5531a9db212cc480991cfc04ae5e61a2b3a8a01650fbb7dc2b88a7"} Jul 20 18:02:07.896943 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:07.896905 2567 generic.go:356] "Generic (PLEG): container finished" podID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerID="5dd36f5a8809a8c35b67ae4b7888b36ac57fb435b256640a574cb7b26811b3a8" exitCode=0 Jul 20 18:02:07.897397 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:07.896997 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" event={"ID":"05df6dcd-1898-4a72-9d7c-10371ad668cf","Type":"ContainerDied","Data":"5dd36f5a8809a8c35b67ae4b7888b36ac57fb435b256640a574cb7b26811b3a8"} Jul 20 18:02:08.904657 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:08.904613 2567 generic.go:356] "Generic (PLEG): container finished" podID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerID="7c4a8e323ba2eeaa351638fff6d35a4e1bb36f01e408a455981103a4d4e065db" exitCode=0 Jul 20 18:02:08.905113 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:08.904704 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" event={"ID":"05df6dcd-1898-4a72-9d7c-10371ad668cf","Type":"ContainerDied","Data":"7c4a8e323ba2eeaa351638fff6d35a4e1bb36f01e408a455981103a4d4e065db"} Jul 20 18:02:09.910464 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:09.910428 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" event={"ID":"0aaedb57-8008-4075-996f-4de38cfb474c","Type":"ContainerStarted","Data":"907609ee683e03ea9ec17f4fd10758c091df39cb696be8b878abfbefb18abf86"} Jul 20 18:02:09.933447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:09.933382 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n56ls" podStartSLOduration=2.285208307 podStartE2EDuration="4.933358905s" totalImagesPullingTime="2.648150598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:02:09.930436502 +0000 UTC m=+359.058553668" watchObservedRunningTime="2026-07-20 18:02:09.933358905 +0000 UTC m=+359.061476074" Jul 20 18:02:10.053304 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.053280 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:10.078028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.078000 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbtfj\" (UniqueName: \"kubernetes.io/projected/05df6dcd-1898-4a72-9d7c-10371ad668cf-kube-api-access-gbtfj\") pod \"05df6dcd-1898-4a72-9d7c-10371ad668cf\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " Jul 20 18:02:10.078173 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.078059 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-util\") pod \"05df6dcd-1898-4a72-9d7c-10371ad668cf\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " Jul 20 18:02:10.078173 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.078135 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-bundle\") pod \"05df6dcd-1898-4a72-9d7c-10371ad668cf\" (UID: \"05df6dcd-1898-4a72-9d7c-10371ad668cf\") " Jul 20 18:02:10.078540 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.078500 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-bundle" (OuterVolumeSpecName: "bundle") pod "05df6dcd-1898-4a72-9d7c-10371ad668cf" (UID: "05df6dcd-1898-4a72-9d7c-10371ad668cf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:02:10.080615 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.080579 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05df6dcd-1898-4a72-9d7c-10371ad668cf-kube-api-access-gbtfj" (OuterVolumeSpecName: "kube-api-access-gbtfj") pod "05df6dcd-1898-4a72-9d7c-10371ad668cf" (UID: "05df6dcd-1898-4a72-9d7c-10371ad668cf"). InnerVolumeSpecName "kube-api-access-gbtfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:02:10.085087 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.085056 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-util" (OuterVolumeSpecName: "util") pod "05df6dcd-1898-4a72-9d7c-10371ad668cf" (UID: "05df6dcd-1898-4a72-9d7c-10371ad668cf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:02:10.179391 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.179289 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:02:10.179391 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.179332 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbtfj\" (UniqueName: \"kubernetes.io/projected/05df6dcd-1898-4a72-9d7c-10371ad668cf-kube-api-access-gbtfj\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:02:10.179391 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.179343 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05df6dcd-1898-4a72-9d7c-10371ad668cf-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:02:10.917076 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.917047 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" Jul 20 18:02:10.917584 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.917079 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7szmm" event={"ID":"05df6dcd-1898-4a72-9d7c-10371ad668cf","Type":"ContainerDied","Data":"0bb48a6b4afd013c3ead57970734c1511aaf6e6d4a0866d09ccaafbe856357fb"} Jul 20 18:02:10.917584 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:10.917132 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bb48a6b4afd013c3ead57970734c1511aaf6e6d4a0866d09ccaafbe856357fb" Jul 20 18:02:24.805289 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805254 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f485h"] Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805574 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="util" Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805585 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="util" Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805594 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="pull" Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805600 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="pull" Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805606 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="extract" Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805611 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="extract" Jul 20 18:02:24.805688 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.805667 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="05df6dcd-1898-4a72-9d7c-10371ad668cf" containerName="extract" Jul 20 18:02:24.812275 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.812252 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:24.815244 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.815218 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zxvph\"" Jul 20 18:02:24.819435 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.819409 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f485h"] Jul 20 18:02:24.903751 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.903705 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frxdr\" (UniqueName: \"kubernetes.io/projected/76e9f174-ccc1-4cc5-82fa-513914c4f983-kube-api-access-frxdr\") pod \"cert-manager-55b4d954d5-f485h\" (UID: \"76e9f174-ccc1-4cc5-82fa-513914c4f983\") " pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:24.903751 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:24.903753 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76e9f174-ccc1-4cc5-82fa-513914c4f983-bound-sa-token\") pod \"cert-manager-55b4d954d5-f485h\" (UID: \"76e9f174-ccc1-4cc5-82fa-513914c4f983\") " pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:25.004558 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.004517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frxdr\" (UniqueName: \"kubernetes.io/projected/76e9f174-ccc1-4cc5-82fa-513914c4f983-kube-api-access-frxdr\") pod \"cert-manager-55b4d954d5-f485h\" (UID: \"76e9f174-ccc1-4cc5-82fa-513914c4f983\") " pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:25.004739 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.004564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76e9f174-ccc1-4cc5-82fa-513914c4f983-bound-sa-token\") pod \"cert-manager-55b4d954d5-f485h\" (UID: \"76e9f174-ccc1-4cc5-82fa-513914c4f983\") " pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:25.013125 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.013094 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76e9f174-ccc1-4cc5-82fa-513914c4f983-bound-sa-token\") pod \"cert-manager-55b4d954d5-f485h\" (UID: \"76e9f174-ccc1-4cc5-82fa-513914c4f983\") " pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:25.013261 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.013127 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frxdr\" (UniqueName: \"kubernetes.io/projected/76e9f174-ccc1-4cc5-82fa-513914c4f983-kube-api-access-frxdr\") pod \"cert-manager-55b4d954d5-f485h\" (UID: \"76e9f174-ccc1-4cc5-82fa-513914c4f983\") " pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:25.122197 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.122084 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f485h" Jul 20 18:02:25.270492 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.270464 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f485h"] Jul 20 18:02:25.273021 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:25.272993 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76e9f174_ccc1_4cc5_82fa_513914c4f983.slice/crio-b37369547bb3fde410af1ae8f9ccf28014497f012570d5af6a8db5a942338cb5 WatchSource:0}: Error finding container b37369547bb3fde410af1ae8f9ccf28014497f012570d5af6a8db5a942338cb5: Status 404 returned error can't find the container with id b37369547bb3fde410af1ae8f9ccf28014497f012570d5af6a8db5a942338cb5 Jul 20 18:02:25.983901 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.983864 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f485h" event={"ID":"76e9f174-ccc1-4cc5-82fa-513914c4f983","Type":"ContainerStarted","Data":"a92945b7b9c602f7b3971c69c1ddcaba8d47dbc425a47680f9577690db20b4c9"} Jul 20 18:02:25.983901 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:25.983912 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f485h" event={"ID":"76e9f174-ccc1-4cc5-82fa-513914c4f983","Type":"ContainerStarted","Data":"b37369547bb3fde410af1ae8f9ccf28014497f012570d5af6a8db5a942338cb5"} Jul 20 18:02:26.006889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.006838 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-f485h" podStartSLOduration=2.006822488 podStartE2EDuration="2.006822488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:02:26.0049807 +0000 UTC m=+375.133097889" watchObservedRunningTime="2026-07-20 18:02:26.006822488 +0000 UTC m=+375.134939654" Jul 20 18:02:26.591029 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.590989 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f"] Jul 20 18:02:26.601877 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.601859 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.604980 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.604950 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nmt99\"" Jul 20 18:02:26.605094 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.605018 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 18:02:26.606098 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.606083 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 18:02:26.618366 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.618344 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f"] Jul 20 18:02:26.719254 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.719215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb6ff\" (UniqueName: \"kubernetes.io/projected/5deacc7d-970a-444c-9a7f-6162965cc47b-kube-api-access-zb6ff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.719454 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.719324 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.719454 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.719373 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.820333 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.820295 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb6ff\" (UniqueName: \"kubernetes.io/projected/5deacc7d-970a-444c-9a7f-6162965cc47b-kube-api-access-zb6ff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.820543 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.820357 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.820543 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.820379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.820708 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.820691 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.820780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.820757 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.829522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.829486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb6ff\" (UniqueName: \"kubernetes.io/projected/5deacc7d-970a-444c-9a7f-6162965cc47b-kube-api-access-zb6ff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:26.911366 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:26.911287 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:27.055836 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:27.055811 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f"] Jul 20 18:02:27.058439 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:27.058404 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5deacc7d_970a_444c_9a7f_6162965cc47b.slice/crio-9223a8efd9738beed435e27948a5b6af70c7344e244e0b47c4382ecbfeec69ba WatchSource:0}: Error finding container 9223a8efd9738beed435e27948a5b6af70c7344e244e0b47c4382ecbfeec69ba: Status 404 returned error can't find the container with id 9223a8efd9738beed435e27948a5b6af70c7344e244e0b47c4382ecbfeec69ba Jul 20 18:02:27.994160 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:27.994125 2567 generic.go:356] "Generic (PLEG): container finished" podID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerID="58db8819818a310ace53b870eaf998160053b25a67e94e2ed66da66917464a7a" exitCode=0 Jul 20 18:02:27.994343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:27.994221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" event={"ID":"5deacc7d-970a-444c-9a7f-6162965cc47b","Type":"ContainerDied","Data":"58db8819818a310ace53b870eaf998160053b25a67e94e2ed66da66917464a7a"} Jul 20 18:02:27.994343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:27.994263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" event={"ID":"5deacc7d-970a-444c-9a7f-6162965cc47b","Type":"ContainerStarted","Data":"9223a8efd9738beed435e27948a5b6af70c7344e244e0b47c4382ecbfeec69ba"} Jul 20 18:02:29.000311 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:29.000277 2567 generic.go:356] "Generic (PLEG): container finished" podID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerID="b65dc82a770f76635c89013fae6f4e6f5b3136dd4dbbeac36c5b3b58fa347343" exitCode=0 Jul 20 18:02:29.000701 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:29.000355 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" event={"ID":"5deacc7d-970a-444c-9a7f-6162965cc47b","Type":"ContainerDied","Data":"b65dc82a770f76635c89013fae6f4e6f5b3136dd4dbbeac36c5b3b58fa347343"} Jul 20 18:02:30.007054 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:30.007018 2567 generic.go:356] "Generic (PLEG): container finished" podID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerID="315808c13abdefb54bb94349afcf37795ff5a83b82de2a412a96c55f55c64cea" exitCode=0 Jul 20 18:02:30.007458 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:30.007093 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" event={"ID":"5deacc7d-970a-444c-9a7f-6162965cc47b","Type":"ContainerDied","Data":"315808c13abdefb54bb94349afcf37795ff5a83b82de2a412a96c55f55c64cea"} Jul 20 18:02:31.136690 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.136663 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:31.258572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.258541 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-bundle\") pod \"5deacc7d-970a-444c-9a7f-6162965cc47b\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " Jul 20 18:02:31.258572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.258580 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-util\") pod \"5deacc7d-970a-444c-9a7f-6162965cc47b\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " Jul 20 18:02:31.258760 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.258633 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb6ff\" (UniqueName: \"kubernetes.io/projected/5deacc7d-970a-444c-9a7f-6162965cc47b-kube-api-access-zb6ff\") pod \"5deacc7d-970a-444c-9a7f-6162965cc47b\" (UID: \"5deacc7d-970a-444c-9a7f-6162965cc47b\") " Jul 20 18:02:31.259306 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.259271 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-bundle" (OuterVolumeSpecName: "bundle") pod "5deacc7d-970a-444c-9a7f-6162965cc47b" (UID: "5deacc7d-970a-444c-9a7f-6162965cc47b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:02:31.260884 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.260858 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5deacc7d-970a-444c-9a7f-6162965cc47b-kube-api-access-zb6ff" (OuterVolumeSpecName: "kube-api-access-zb6ff") pod "5deacc7d-970a-444c-9a7f-6162965cc47b" (UID: "5deacc7d-970a-444c-9a7f-6162965cc47b"). InnerVolumeSpecName "kube-api-access-zb6ff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:02:31.264220 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.264147 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-util" (OuterVolumeSpecName: "util") pod "5deacc7d-970a-444c-9a7f-6162965cc47b" (UID: "5deacc7d-970a-444c-9a7f-6162965cc47b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:02:31.359500 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.359469 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:02:31.359500 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.359493 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5deacc7d-970a-444c-9a7f-6162965cc47b-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:02:31.359500 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:31.359503 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb6ff\" (UniqueName: \"kubernetes.io/projected/5deacc7d-970a-444c-9a7f-6162965cc47b-kube-api-access-zb6ff\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:02:32.018380 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:32.018341 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" event={"ID":"5deacc7d-970a-444c-9a7f-6162965cc47b","Type":"ContainerDied","Data":"9223a8efd9738beed435e27948a5b6af70c7344e244e0b47c4382ecbfeec69ba"} Jul 20 18:02:32.018548 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:32.018398 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9223a8efd9738beed435e27948a5b6af70c7344e244e0b47c4382ecbfeec69ba" Jul 20 18:02:32.018548 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:32.018360 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgd89f" Jul 20 18:02:38.074298 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074262 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms"] Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074581 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="util" Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074592 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="util" Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074605 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="pull" Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074610 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="pull" Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074618 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="extract" Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074624 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="extract" Jul 20 18:02:38.074745 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.074677 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5deacc7d-970a-444c-9a7f-6162965cc47b" containerName="extract" Jul 20 18:02:38.076946 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.076931 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.080350 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.080331 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 18:02:38.081117 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.081102 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 18:02:38.082150 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.082133 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4q2sn\"" Jul 20 18:02:38.082421 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.082404 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 18:02:38.082514 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.082496 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 18:02:38.109671 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.109639 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms"] Jul 20 18:02:38.218617 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.218581 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8svx8\" (UniqueName: \"kubernetes.io/projected/e190c526-deae-442b-bb20-7edbafa87611-kube-api-access-8svx8\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.218788 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.218646 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e190c526-deae-442b-bb20-7edbafa87611-tmp\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.218788 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.218707 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e190c526-deae-442b-bb20-7edbafa87611-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.218788 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.218752 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e190c526-deae-442b-bb20-7edbafa87611-webhook-cert\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.218788 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.218778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e190c526-deae-442b-bb20-7edbafa87611-manifests\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.319606 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.319561 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8svx8\" (UniqueName: \"kubernetes.io/projected/e190c526-deae-442b-bb20-7edbafa87611-kube-api-access-8svx8\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.319789 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.319629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e190c526-deae-442b-bb20-7edbafa87611-tmp\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.319789 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.319721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e190c526-deae-442b-bb20-7edbafa87611-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.319789 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.319756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e190c526-deae-442b-bb20-7edbafa87611-webhook-cert\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.319938 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.319791 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e190c526-deae-442b-bb20-7edbafa87611-manifests\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.320258 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.320227 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e190c526-deae-442b-bb20-7edbafa87611-manifests\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.321869 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.321835 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e190c526-deae-442b-bb20-7edbafa87611-tmp\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.322271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.322254 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e190c526-deae-442b-bb20-7edbafa87611-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.322325 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.322287 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e190c526-deae-442b-bb20-7edbafa87611-webhook-cert\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.328885 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.328832 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8svx8\" (UniqueName: \"kubernetes.io/projected/e190c526-deae-442b-bb20-7edbafa87611-kube-api-access-8svx8\") pod \"opendatahub-operator-controller-manager-79747b4d4c-dqbms\" (UID: \"e190c526-deae-442b-bb20-7edbafa87611\") " pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.387786 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.387749 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:38.527695 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:38.527671 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms"] Jul 20 18:02:38.530535 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:38.530501 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode190c526_deae_442b_bb20_7edbafa87611.slice/crio-a8bf4b6ae67e174049a6872b200be398bb1e3d00ea60b1706eabc08dd4a148d5 WatchSource:0}: Error finding container a8bf4b6ae67e174049a6872b200be398bb1e3d00ea60b1706eabc08dd4a148d5: Status 404 returned error can't find the container with id a8bf4b6ae67e174049a6872b200be398bb1e3d00ea60b1706eabc08dd4a148d5 Jul 20 18:02:39.049489 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:39.049452 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" event={"ID":"e190c526-deae-442b-bb20-7edbafa87611","Type":"ContainerStarted","Data":"a8bf4b6ae67e174049a6872b200be398bb1e3d00ea60b1706eabc08dd4a148d5"} Jul 20 18:02:42.066172 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:42.066135 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" event={"ID":"e190c526-deae-442b-bb20-7edbafa87611","Type":"ContainerStarted","Data":"1b67d21ce68c3d3fc82b289ba53cd3593d7858c9679e2a2c917d7bd6ecec9148"} Jul 20 18:02:43.072059 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:43.072025 2567 generic.go:356] "Generic (PLEG): container finished" podID="e190c526-deae-442b-bb20-7edbafa87611" containerID="1b67d21ce68c3d3fc82b289ba53cd3593d7858c9679e2a2c917d7bd6ecec9148" exitCode=0 Jul 20 18:02:43.072550 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:43.072064 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" event={"ID":"e190c526-deae-442b-bb20-7edbafa87611","Type":"ContainerDied","Data":"1b67d21ce68c3d3fc82b289ba53cd3593d7858c9679e2a2c917d7bd6ecec9148"} Jul 20 18:02:44.079577 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:44.079540 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" event={"ID":"e190c526-deae-442b-bb20-7edbafa87611","Type":"ContainerStarted","Data":"c1fa72e05abe70644d39f828aa6861aaf70d56e6e1a4268a07bb6b8435016794"} Jul 20 18:02:44.080067 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:44.079649 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:44.105255 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:44.105205 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" podStartSLOduration=1.463294246 podStartE2EDuration="6.105171833s" totalImagesPullingTime="3.504049938s" totalInitContainerRuntime="1.137827649s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:02:44.103345666 +0000 UTC m=+393.231462832" watchObservedRunningTime="2026-07-20 18:02:44.105171833 +0000 UTC m=+393.233289000" Jul 20 18:02:55.087321 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:55.087292 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79747b4d4c-dqbms" Jul 20 18:02:58.023680 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.023637 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28"] Jul 20 18:02:58.029573 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.029544 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.034482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.034432 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fpgjs\"" Jul 20 18:02:58.034482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.034470 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 18:02:58.034672 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.034464 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 18:02:58.034672 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.034523 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 18:02:58.034790 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.034773 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 18:02:58.058890 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.058864 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28"] Jul 20 18:02:58.092564 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.092528 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5d8d836-6592-4308-ac78-e6bd720f66fd-tls-certs\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.092564 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.092573 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d8d836-6592-4308-ac78-e6bd720f66fd-tmp\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.092756 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.092614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6n8f\" (UniqueName: \"kubernetes.io/projected/e5d8d836-6592-4308-ac78-e6bd720f66fd-kube-api-access-k6n8f\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.193549 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.193514 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5d8d836-6592-4308-ac78-e6bd720f66fd-tls-certs\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.193732 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.193560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d8d836-6592-4308-ac78-e6bd720f66fd-tmp\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.193732 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.193594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6n8f\" (UniqueName: \"kubernetes.io/projected/e5d8d836-6592-4308-ac78-e6bd720f66fd-kube-api-access-k6n8f\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.196069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.196043 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d8d836-6592-4308-ac78-e6bd720f66fd-tmp\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.196258 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.196238 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5d8d836-6592-4308-ac78-e6bd720f66fd-tls-certs\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.205330 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.205308 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6n8f\" (UniqueName: \"kubernetes.io/projected/e5d8d836-6592-4308-ac78-e6bd720f66fd-kube-api-access-k6n8f\") pod \"kube-auth-proxy-75cc6b94bb-szq28\" (UID: \"e5d8d836-6592-4308-ac78-e6bd720f66fd\") " pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.284736 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.284660 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-drdk6"] Jul 20 18:02:58.287114 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.287092 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.289953 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.289915 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 20 18:02:58.290164 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.290140 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4mqw7\"" Jul 20 18:02:58.294145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.294119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwtlb\" (UniqueName: \"kubernetes.io/projected/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-kube-api-access-xwtlb\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.294274 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.294161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-cert\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.307765 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.307736 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-drdk6"] Jul 20 18:02:58.339458 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.339432 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" Jul 20 18:02:58.395294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.395259 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwtlb\" (UniqueName: \"kubernetes.io/projected/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-kube-api-access-xwtlb\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.395451 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.395309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-cert\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.395944 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:02:58.395538 2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 20 18:02:58.395944 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:02:58.395634 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-cert podName:1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4 nodeName:}" failed. No retries permitted until 2026-07-20 18:02:58.895611153 +0000 UTC m=+408.023728300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-cert") pod "odh-model-controller-7998c67dc4-drdk6" (UID: "1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4") : secret "odh-model-controller-webhook-cert" not found Jul 20 18:02:58.413463 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.413436 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwtlb\" (UniqueName: \"kubernetes.io/projected/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-kube-api-access-xwtlb\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.485014 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.484990 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28"] Jul 20 18:02:58.486906 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:58.486869 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d8d836_6592_4308_ac78_e6bd720f66fd.slice/crio-08bf87cb1de4904fdf20af52d1ed4da0f9682b201ec6c755f3c432a7632d51c5 WatchSource:0}: Error finding container 08bf87cb1de4904fdf20af52d1ed4da0f9682b201ec6c755f3c432a7632d51c5: Status 404 returned error can't find the container with id 08bf87cb1de4904fdf20af52d1ed4da0f9682b201ec6c755f3c432a7632d51c5 Jul 20 18:02:58.898561 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.898521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-cert\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:58.900880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:58.900849 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4-cert\") pod \"odh-model-controller-7998c67dc4-drdk6\" (UID: \"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:59.152844 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.152754 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" event={"ID":"e5d8d836-6592-4308-ac78-e6bd720f66fd","Type":"ContainerStarted","Data":"08bf87cb1de4904fdf20af52d1ed4da0f9682b201ec6c755f3c432a7632d51c5"} Jul 20 18:02:59.199465 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.199119 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:02:59.375014 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.374678 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-drdk6"] Jul 20 18:02:59.378603 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:59.378573 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4cfa7a_f02a_4713_bc09_4e5f4bd44dd4.slice/crio-333a6e67c02d144bc30fc0dc67a3e6dc3874f77100fb8c7f9de70bde436a705e WatchSource:0}: Error finding container 333a6e67c02d144bc30fc0dc67a3e6dc3874f77100fb8c7f9de70bde436a705e: Status 404 returned error can't find the container with id 333a6e67c02d144bc30fc0dc67a3e6dc3874f77100fb8c7f9de70bde436a705e Jul 20 18:02:59.466828 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.466744 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mwntz"] Jul 20 18:02:59.470390 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.470359 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.474362 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.474310 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 20 18:02:59.476266 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.476244 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nmg8c\"" Jul 20 18:02:59.495153 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.495123 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mwntz"] Jul 20 18:02:59.503016 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.502982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mj6\" (UniqueName: \"kubernetes.io/projected/dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e-kube-api-access-n4mj6\") pod \"model-serving-api-54d89bd5fc-mwntz\" (UID: \"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.503159 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.503037 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mwntz\" (UID: \"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.603516 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.603482 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mj6\" (UniqueName: \"kubernetes.io/projected/dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e-kube-api-access-n4mj6\") pod \"model-serving-api-54d89bd5fc-mwntz\" (UID: \"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.603516 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.603522 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mwntz\" (UID: \"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.605887 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.605861 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mwntz\" (UID: \"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.614856 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.614830 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mj6\" (UniqueName: \"kubernetes.io/projected/dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e-kube-api-access-n4mj6\") pod \"model-serving-api-54d89bd5fc-mwntz\" (UID: \"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.784528 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.784411 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:02:59.803386 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.803344 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n95ps"] Jul 20 18:02:59.808165 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.808139 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:02:59.811966 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.811944 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 20 18:02:59.812474 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.812452 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-55w2v\"" Jul 20 18:02:59.812671 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.812654 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 18:02:59.812729 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.812713 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 20 18:02:59.812779 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.812727 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 20 18:02:59.820294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.820254 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n95ps"] Jul 20 18:02:59.937970 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:02:59.937908 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mwntz"] Jul 20 18:02:59.940128 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:02:59.940101 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd3ffc2_64b6_48dc_81a7_3ef3a0cdff2e.slice/crio-67374b68b7d5fcbcdb24e49197efbcce60a8808271bd148bb5a35777f0a2dfb7 WatchSource:0}: Error finding container 67374b68b7d5fcbcdb24e49197efbcce60a8808271bd148bb5a35777f0a2dfb7: Status 404 returned error can't find the container with id 67374b68b7d5fcbcdb24e49197efbcce60a8808271bd148bb5a35777f0a2dfb7 Jul 20 18:03:00.007349 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007313 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-istio-token\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.007522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007364 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.007522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007434 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvzks\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-kube-api-access-gvzks\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.007522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.007522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007494 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88950160-ea24-4486-b958-a24e29f35bb0-local-certs\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.007522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007508 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.007681 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.007601 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-cacerts\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108626 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108587 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108626 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108627 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88950160-ea24-4486-b958-a24e29f35bb0-local-certs\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108869 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108643 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108869 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108681 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-cacerts\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108869 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108730 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-istio-token\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108869 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.108869 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.108793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvzks\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-kube-api-access-gvzks\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.109448 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.109409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.111307 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.111274 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.111432 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.111276 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88950160-ea24-4486-b958-a24e29f35bb0-local-certs\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.111432 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.111358 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-cacerts\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.111539 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.111428 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.117269 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.117166 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-istio-token\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.117465 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.117447 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvzks\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-kube-api-access-gvzks\") pod \"istiod-openshift-gateway-7574459656-n95ps\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.122287 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.122236 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:00.164764 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.164678 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" event={"ID":"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4","Type":"ContainerStarted","Data":"333a6e67c02d144bc30fc0dc67a3e6dc3874f77100fb8c7f9de70bde436a705e"} Jul 20 18:03:00.167611 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.167543 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" event={"ID":"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e","Type":"ContainerStarted","Data":"67374b68b7d5fcbcdb24e49197efbcce60a8808271bd148bb5a35777f0a2dfb7"} Jul 20 18:03:00.317439 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:00.317400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n95ps"] Jul 20 18:03:00.324077 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:03:00.323157 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88950160_ea24_4486_b958_a24e29f35bb0.slice/crio-be966df0b783cc93aa89960aabebb58087edc8de8c2ab01d572b478378b84416 WatchSource:0}: Error finding container be966df0b783cc93aa89960aabebb58087edc8de8c2ab01d572b478378b84416: Status 404 returned error can't find the container with id be966df0b783cc93aa89960aabebb58087edc8de8c2ab01d572b478378b84416 Jul 20 18:03:01.176017 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:01.175951 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" event={"ID":"88950160-ea24-4486-b958-a24e29f35bb0","Type":"ContainerStarted","Data":"be966df0b783cc93aa89960aabebb58087edc8de8c2ab01d572b478378b84416"} Jul 20 18:03:04.195772 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.195732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" event={"ID":"1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4","Type":"ContainerStarted","Data":"4bcc7d8a17ea6f9f2a12d1e8f439dd04efeb84a14c385a1a2e574d4dc995ad91"} Jul 20 18:03:04.196281 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.195879 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:03:04.197831 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.197801 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" event={"ID":"dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e","Type":"ContainerStarted","Data":"866d5cec2ffc61fa0b3982fd54f66afeee2799db975e1df7404885a30b1a48f9"} Jul 20 18:03:04.197945 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.197874 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:03:04.199734 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.199710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" event={"ID":"e5d8d836-6592-4308-ac78-e6bd720f66fd","Type":"ContainerStarted","Data":"42927cbae1ce1d38fd1f3a5a4e3c74c62cd3d24f5e92abe40abc812c9f2c177f"} Jul 20 18:03:04.214339 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.214298 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" podStartSLOduration=2.243329543 podStartE2EDuration="6.21428485s" totalImagesPullingTime="3.970955307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:04.213763649 +0000 UTC m=+413.341880816" watchObservedRunningTime="2026-07-20 18:03:04.21428485 +0000 UTC m=+413.342401994" Jul 20 18:03:04.231880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:04.231832 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" podStartSLOduration=1.836976425 podStartE2EDuration="5.23181297s" totalImagesPullingTime="3.394836545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:04.230758113 +0000 UTC m=+413.358875304" watchObservedRunningTime="2026-07-20 18:03:04.23181297 +0000 UTC m=+413.359930139" Jul 20 18:03:05.944542 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:05.944493 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:03:05.944884 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:05.944582 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:03:06.212369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:06.212271 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" event={"ID":"88950160-ea24-4486-b958-a24e29f35bb0","Type":"ContainerStarted","Data":"af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3"} Jul 20 18:03:06.212567 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:06.212394 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:06.237645 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:06.237597 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" podStartSLOduration=1.6193109940000001 podStartE2EDuration="7.237583206s" totalImagesPullingTime="5.618272212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:06.235500313 +0000 UTC m=+415.363617479" watchObservedRunningTime="2026-07-20 18:03:06.237583206 +0000 UTC m=+415.365700372" Jul 20 18:03:06.238351 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:06.238321 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75cc6b94bb-szq28" podStartSLOduration=4.390922839 podStartE2EDuration="9.238313207s" totalImagesPullingTime="4.847390368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:04.247603212 +0000 UTC m=+413.375720403" watchObservedRunningTime="2026-07-20 18:03:06.238313207 +0000 UTC m=+415.366430373" Jul 20 18:03:07.220071 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:07.220032 2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-n95ps container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 18:03:07.220476 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:07.220099 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" podUID="88950160-ea24-4486-b958-a24e29f35bb0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 18:03:08.097837 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.097806 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47"] Jul 20 18:03:08.100582 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.100565 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.108616 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.108594 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p4msj\"" Jul 20 18:03:08.131013 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.130978 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47"] Jul 20 18:03:08.181430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181395 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181615 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qhs\" (UniqueName: \"kubernetes.io/projected/57aadb03-24ae-414a-80d7-6aa445139129-kube-api-access-v6qhs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181615 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181479 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57aadb03-24ae-414a-80d7-6aa445139129-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181615 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181550 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57aadb03-24ae-414a-80d7-6aa445139129-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181615 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181777 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181621 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181777 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181659 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181777 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181680 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57aadb03-24ae-414a-80d7-6aa445139129-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.181777 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.181749 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.282897 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.282860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.282909 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.282937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qhs\" (UniqueName: \"kubernetes.io/projected/57aadb03-24ae-414a-80d7-6aa445139129-kube-api-access-v6qhs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.282961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57aadb03-24ae-414a-80d7-6aa445139129-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.282989 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57aadb03-24ae-414a-80d7-6aa445139129-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283126 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283371 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57aadb03-24ae-414a-80d7-6aa445139129-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283753 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283398 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283753 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283450 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283753 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283584 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.283753 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.283632 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.284120 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.284096 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57aadb03-24ae-414a-80d7-6aa445139129-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.285269 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.285246 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57aadb03-24ae-414a-80d7-6aa445139129-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.285496 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.285480 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57aadb03-24ae-414a-80d7-6aa445139129-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.306961 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.306931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57aadb03-24ae-414a-80d7-6aa445139129-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.308285 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.308259 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qhs\" (UniqueName: \"kubernetes.io/projected/57aadb03-24ae-414a-80d7-6aa445139129-kube-api-access-v6qhs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4d7j47\" (UID: \"57aadb03-24ae-414a-80d7-6aa445139129\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.410316 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.410232 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:08.577075 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:08.577045 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47"] Jul 20 18:03:08.579505 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:03:08.579467 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57aadb03_24ae_414a_80d7_6aa445139129.slice/crio-c57a69ba350e0aaeb679d4e7ed97b240c3952ce2778abeadb4c0a2096d71c711 WatchSource:0}: Error finding container c57a69ba350e0aaeb679d4e7ed97b240c3952ce2778abeadb4c0a2096d71c711: Status 404 returned error can't find the container with id c57a69ba350e0aaeb679d4e7ed97b240c3952ce2778abeadb4c0a2096d71c711 Jul 20 18:03:09.229412 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:09.229371 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" event={"ID":"57aadb03-24ae-414a-80d7-6aa445139129","Type":"ContainerStarted","Data":"c57a69ba350e0aaeb679d4e7ed97b240c3952ce2778abeadb4c0a2096d71c711"} Jul 20 18:03:10.211042 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.211007 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj"] Jul 20 18:03:10.216287 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.216262 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.218920 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.218895 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 20 18:03:10.219361 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.219339 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:10.219475 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.219400 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6d9kk\"" Jul 20 18:03:10.232297 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.232263 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj"] Jul 20 18:03:10.235681 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.235654 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv"] Jul 20 18:03:10.240302 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.240279 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.242880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.242859 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 20 18:03:10.242980 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.242899 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-k5m4h\"" Jul 20 18:03:10.249157 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.249120 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv"] Jul 20 18:03:10.302406 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.302342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c757ec3-9d16-4264-97d2-4758e47d8703-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2gdzv\" (UID: \"0c757ec3-9d16-4264-97d2-4758e47d8703\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.302668 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.302417 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq42m\" (UniqueName: \"kubernetes.io/projected/5d4341e7-78f1-442b-aabb-6b381486e766-kube-api-access-fq42m\") pod \"kserve-controller-manager-d6f4b65b8-gx8bj\" (UID: \"5d4341e7-78f1-442b-aabb-6b381486e766\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.302668 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.302588 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d4341e7-78f1-442b-aabb-6b381486e766-cert\") pod \"kserve-controller-manager-d6f4b65b8-gx8bj\" (UID: \"5d4341e7-78f1-442b-aabb-6b381486e766\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.302817 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.302746 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkhj7\" (UniqueName: \"kubernetes.io/projected/0c757ec3-9d16-4264-97d2-4758e47d8703-kube-api-access-wkhj7\") pod \"llmisvc-controller-manager-777c6bd7d5-2gdzv\" (UID: \"0c757ec3-9d16-4264-97d2-4758e47d8703\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.403531 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.403497 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkhj7\" (UniqueName: \"kubernetes.io/projected/0c757ec3-9d16-4264-97d2-4758e47d8703-kube-api-access-wkhj7\") pod \"llmisvc-controller-manager-777c6bd7d5-2gdzv\" (UID: \"0c757ec3-9d16-4264-97d2-4758e47d8703\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.403730 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.403551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c757ec3-9d16-4264-97d2-4758e47d8703-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2gdzv\" (UID: \"0c757ec3-9d16-4264-97d2-4758e47d8703\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.403730 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.403677 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq42m\" (UniqueName: \"kubernetes.io/projected/5d4341e7-78f1-442b-aabb-6b381486e766-kube-api-access-fq42m\") pod \"kserve-controller-manager-d6f4b65b8-gx8bj\" (UID: \"5d4341e7-78f1-442b-aabb-6b381486e766\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.403847 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.403733 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d4341e7-78f1-442b-aabb-6b381486e766-cert\") pod \"kserve-controller-manager-d6f4b65b8-gx8bj\" (UID: \"5d4341e7-78f1-442b-aabb-6b381486e766\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.406196 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.406157 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d4341e7-78f1-442b-aabb-6b381486e766-cert\") pod \"kserve-controller-manager-d6f4b65b8-gx8bj\" (UID: \"5d4341e7-78f1-442b-aabb-6b381486e766\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.406308 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.406157 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c757ec3-9d16-4264-97d2-4758e47d8703-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2gdzv\" (UID: \"0c757ec3-9d16-4264-97d2-4758e47d8703\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.415773 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.415743 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkhj7\" (UniqueName: \"kubernetes.io/projected/0c757ec3-9d16-4264-97d2-4758e47d8703-kube-api-access-wkhj7\") pod \"llmisvc-controller-manager-777c6bd7d5-2gdzv\" (UID: \"0c757ec3-9d16-4264-97d2-4758e47d8703\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.415875 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.415794 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq42m\" (UniqueName: \"kubernetes.io/projected/5d4341e7-78f1-442b-aabb-6b381486e766-kube-api-access-fq42m\") pod \"kserve-controller-manager-d6f4b65b8-gx8bj\" (UID: \"5d4341e7-78f1-442b-aabb-6b381486e766\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.532404 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.532326 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:10.554375 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.554348 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:10.711329 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.710751 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj"] Jul 20 18:03:10.758958 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:10.754700 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv"] Jul 20 18:03:10.762904 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:03:10.762876 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c757ec3_9d16_4264_97d2_4758e47d8703.slice/crio-9b40e7e6d6d36de9bc2120195ca82844aacd0374f285e16a4d1dd2af2f9f4333 WatchSource:0}: Error finding container 9b40e7e6d6d36de9bc2120195ca82844aacd0374f285e16a4d1dd2af2f9f4333: Status 404 returned error can't find the container with id 9b40e7e6d6d36de9bc2120195ca82844aacd0374f285e16a4d1dd2af2f9f4333 Jul 20 18:03:11.243317 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:11.243280 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" event={"ID":"0c757ec3-9d16-4264-97d2-4758e47d8703","Type":"ContainerStarted","Data":"9b40e7e6d6d36de9bc2120195ca82844aacd0374f285e16a4d1dd2af2f9f4333"} Jul 20 18:03:11.244283 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:11.244261 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" event={"ID":"5d4341e7-78f1-442b-aabb-6b381486e766","Type":"ContainerStarted","Data":"2564e213aed627fe30ea8d37da1e8fb60309235dc8d529be6a1de78a27d8136c"} Jul 20 18:03:12.498455 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:12.498415 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:03:12.498733 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:12.498548 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:03:12.498733 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:12.498577 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:03:13.258878 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:13.258810 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" event={"ID":"57aadb03-24ae-414a-80d7-6aa445139129","Type":"ContainerStarted","Data":"1d02cc58882fc8241ccae0c8d91b2aeb5d32a46c313296c3fbbceb73542d47b2"} Jul 20 18:03:13.281650 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:13.281592 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" podStartSLOduration=1.365626021 podStartE2EDuration="5.281573346s" totalImagesPullingTime="3.915947325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:13.279461091 +0000 UTC m=+422.407578315" watchObservedRunningTime="2026-07-20 18:03:13.281573346 +0000 UTC m=+422.409690514" Jul 20 18:03:13.411006 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:13.410974 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:13.416621 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:13.416589 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:13.911134 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:13.911109 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n95ps"] Jul 20 18:03:13.911592 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:13.911324 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" podUID="88950160-ea24-4486-b958-a24e29f35bb0" containerName="discovery" containerID="cri-o://af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3" gracePeriod=30 Jul 20 18:03:14.168506 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.168482 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:14.242094 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242058 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-cacerts\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242295 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242128 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88950160-ea24-4486-b958-a24e29f35bb0-local-certs\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242295 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-dns-cert\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242295 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242227 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-kubeconfig\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242322 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-istio-token\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242393 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-ca-configmap\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242454 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvzks\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-kube-api-access-gvzks\") pod \"88950160-ea24-4486-b958-a24e29f35bb0\" (UID: \"88950160-ea24-4486-b958-a24e29f35bb0\") " Jul 20 18:03:14.242855 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.242794 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:03:14.244512 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.244482 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:03:14.244639 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.244547 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-cacerts" (OuterVolumeSpecName: "cacerts") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:03:14.244786 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.244761 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-istio-token" (OuterVolumeSpecName: "istio-token") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:03:14.244843 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.244820 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88950160-ea24-4486-b958-a24e29f35bb0-local-certs" (OuterVolumeSpecName: "local-certs") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:03:14.244903 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.244886 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:03:14.244960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.244945 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-kube-api-access-gvzks" (OuterVolumeSpecName: "kube-api-access-gvzks") pod "88950160-ea24-4486-b958-a24e29f35bb0" (UID: "88950160-ea24-4486-b958-a24e29f35bb0"). InnerVolumeSpecName "kube-api-access-gvzks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:03:14.266206 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.266109 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" event={"ID":"5d4341e7-78f1-442b-aabb-6b381486e766","Type":"ContainerStarted","Data":"9d0114e36c7b1fd30405ba03710c37a1588ae0addd1b69de56aeb22e50333ba9"} Jul 20 18:03:14.266206 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.266173 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:14.267430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.267405 2567 generic.go:356] "Generic (PLEG): container finished" podID="88950160-ea24-4486-b958-a24e29f35bb0" containerID="af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3" exitCode=0 Jul 20 18:03:14.267536 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.267436 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" Jul 20 18:03:14.267536 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.267486 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" event={"ID":"88950160-ea24-4486-b958-a24e29f35bb0","Type":"ContainerDied","Data":"af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3"} Jul 20 18:03:14.267536 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.267530 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n95ps" event={"ID":"88950160-ea24-4486-b958-a24e29f35bb0","Type":"ContainerDied","Data":"be966df0b783cc93aa89960aabebb58087edc8de8c2ab01d572b478378b84416"} Jul 20 18:03:14.267636 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.267549 2567 scope.go:117] "RemoveContainer" containerID="af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3" Jul 20 18:03:14.267899 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.267869 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:14.268992 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.268975 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4d7j47" Jul 20 18:03:14.276638 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.276609 2567 scope.go:117] "RemoveContainer" containerID="af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3" Jul 20 18:03:14.276876 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:03:14.276858 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3\": container with ID starting with af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3 not found: ID does not exist" containerID="af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3" Jul 20 18:03:14.276943 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.276884 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3"} err="failed to get container status \"af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3\": rpc error: code = NotFound desc = could not find container \"af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3\": container with ID starting with af4f8ba602068a743a48af8554ad2aa4e195f8725459457069610645f4f550d3 not found: ID does not exist" Jul 20 18:03:14.293356 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.293314 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" podStartSLOduration=1.123319666 podStartE2EDuration="4.293300497s" totalImagesPullingTime="3.169980831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:14.291243766 +0000 UTC m=+423.419360955" watchObservedRunningTime="2026-07-20 18:03:14.293300497 +0000 UTC m=+423.421417663" Jul 20 18:03:14.344130 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344072 2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88950160-ea24-4486-b958-a24e29f35bb0-local-certs\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.344350 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344200 2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-dns-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.344350 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344340 2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-istio-kubeconfig\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.344482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344364 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-istio-token\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.344482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344382 2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88950160-ea24-4486-b958-a24e29f35bb0-istio-csr-ca-configmap\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.344482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344397 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvzks\" (UniqueName: \"kubernetes.io/projected/88950160-ea24-4486-b958-a24e29f35bb0-kube-api-access-gvzks\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.344482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.344427 2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88950160-ea24-4486-b958-a24e29f35bb0-cacerts\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:03:14.384581 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.384546 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n95ps"] Jul 20 18:03:14.417700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:14.417656 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n95ps"] Jul 20 18:03:15.207523 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:15.207480 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-drdk6" Jul 20 18:03:15.209951 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:15.209930 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mwntz" Jul 20 18:03:15.441788 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:15.441746 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88950160-ea24-4486-b958-a24e29f35bb0" path="/var/lib/kubelet/pods/88950160-ea24-4486-b958-a24e29f35bb0/volumes" Jul 20 18:03:18.292904 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:18.292868 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" event={"ID":"0c757ec3-9d16-4264-97d2-4758e47d8703","Type":"ContainerStarted","Data":"26e2c58f1f86a027eb79d3de58633de435368f09cca712c581761d0e2b71948f"} Jul 20 18:03:18.293342 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:18.292974 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:18.313263 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:18.313216 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" podStartSLOduration=1.515689583 podStartE2EDuration="8.31320209s" totalImagesPullingTime="6.797512507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:03:18.31162804 +0000 UTC m=+427.439745206" watchObservedRunningTime="2026-07-20 18:03:18.31320209 +0000 UTC m=+427.441319282" Jul 20 18:03:45.280977 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:45.280901 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gx8bj" Jul 20 18:03:49.300138 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:49.300099 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2gdzv" Jul 20 18:03:59.118284 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.118249 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jpkj"] Jul 20 18:03:59.118750 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.118609 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88950160-ea24-4486-b958-a24e29f35bb0" containerName="discovery" Jul 20 18:03:59.118750 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.118619 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="88950160-ea24-4486-b958-a24e29f35bb0" containerName="discovery" Jul 20 18:03:59.118750 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.118681 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="88950160-ea24-4486-b958-a24e29f35bb0" containerName="discovery" Jul 20 18:03:59.130124 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.130098 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:03:59.132925 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.132892 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 18:03:59.133064 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.132899 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 18:03:59.133125 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.133069 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2rhvv\"" Jul 20 18:03:59.134730 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.134708 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jpkj"] Jul 20 18:03:59.241776 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.241736 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2zl7\" (UniqueName: \"kubernetes.io/projected/2d509dc5-d38f-4ce1-9dcf-aa724c96e301-kube-api-access-k2zl7\") pod \"kuadrant-operator-catalog-7jpkj\" (UID: \"2d509dc5-d38f-4ce1-9dcf-aa724c96e301\") " pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:03:59.342433 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.342394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2zl7\" (UniqueName: \"kubernetes.io/projected/2d509dc5-d38f-4ce1-9dcf-aa724c96e301-kube-api-access-k2zl7\") pod \"kuadrant-operator-catalog-7jpkj\" (UID: \"2d509dc5-d38f-4ce1-9dcf-aa724c96e301\") " pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:03:59.354397 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.354373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2zl7\" (UniqueName: \"kubernetes.io/projected/2d509dc5-d38f-4ce1-9dcf-aa724c96e301-kube-api-access-k2zl7\") pod \"kuadrant-operator-catalog-7jpkj\" (UID: \"2d509dc5-d38f-4ce1-9dcf-aa724c96e301\") " pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:03:59.441147 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.441072 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:03:59.479265 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.479229 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jpkj"] Jul 20 18:03:59.578136 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.578108 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jpkj"] Jul 20 18:03:59.580354 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:03:59.580325 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d509dc5_d38f_4ce1_9dcf_aa724c96e301.slice/crio-81ba2784b96fc80169e858b3d6467e0d81f1fab42a0d7f52a001243164cb86db WatchSource:0}: Error finding container 81ba2784b96fc80169e858b3d6467e0d81f1fab42a0d7f52a001243164cb86db: Status 404 returned error can't find the container with id 81ba2784b96fc80169e858b3d6467e0d81f1fab42a0d7f52a001243164cb86db Jul 20 18:03:59.689578 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.689541 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lfw48"] Jul 20 18:03:59.692959 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.692914 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:03:59.703522 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.703498 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lfw48"] Jul 20 18:03:59.745651 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.745616 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5bdr\" (UniqueName: \"kubernetes.io/projected/1ccf85ab-fb69-4e69-ba1c-65d079d9086b-kube-api-access-z5bdr\") pod \"kuadrant-operator-catalog-lfw48\" (UID: \"1ccf85ab-fb69-4e69-ba1c-65d079d9086b\") " pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:03:59.846429 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.846402 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5bdr\" (UniqueName: \"kubernetes.io/projected/1ccf85ab-fb69-4e69-ba1c-65d079d9086b-kube-api-access-z5bdr\") pod \"kuadrant-operator-catalog-lfw48\" (UID: \"1ccf85ab-fb69-4e69-ba1c-65d079d9086b\") " pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:03:59.855019 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:03:59.854995 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5bdr\" (UniqueName: \"kubernetes.io/projected/1ccf85ab-fb69-4e69-ba1c-65d079d9086b-kube-api-access-z5bdr\") pod \"kuadrant-operator-catalog-lfw48\" (UID: \"1ccf85ab-fb69-4e69-ba1c-65d079d9086b\") " pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:04:00.004195 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:00.004093 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:04:00.144612 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:00.144582 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lfw48"] Jul 20 18:04:00.146288 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:00.146248 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccf85ab_fb69_4e69_ba1c_65d079d9086b.slice/crio-50255b8b49732e002f621fa85a57f0cd489790f9b8c909e1ddf110b851e0f2de WatchSource:0}: Error finding container 50255b8b49732e002f621fa85a57f0cd489790f9b8c909e1ddf110b851e0f2de: Status 404 returned error can't find the container with id 50255b8b49732e002f621fa85a57f0cd489790f9b8c909e1ddf110b851e0f2de Jul 20 18:04:00.498763 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:00.498671 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" event={"ID":"2d509dc5-d38f-4ce1-9dcf-aa724c96e301","Type":"ContainerStarted","Data":"81ba2784b96fc80169e858b3d6467e0d81f1fab42a0d7f52a001243164cb86db"} Jul 20 18:04:00.500993 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:00.500951 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" event={"ID":"1ccf85ab-fb69-4e69-ba1c-65d079d9086b","Type":"ContainerStarted","Data":"50255b8b49732e002f621fa85a57f0cd489790f9b8c909e1ddf110b851e0f2de"} Jul 20 18:04:02.513067 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.513033 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" event={"ID":"2d509dc5-d38f-4ce1-9dcf-aa724c96e301","Type":"ContainerStarted","Data":"d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2"} Jul 20 18:04:02.513547 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.513077 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" podUID="2d509dc5-d38f-4ce1-9dcf-aa724c96e301" containerName="registry-server" containerID="cri-o://d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2" gracePeriod=2 Jul 20 18:04:02.530119 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.530076 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" podStartSLOduration=1.216915297 podStartE2EDuration="3.530058813s" totalImagesPullingTime="2.313143516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:04:02.528468625 +0000 UTC m=+471.656585792" watchObservedRunningTime="2026-07-20 18:04:02.530058813 +0000 UTC m=+471.658175979" Jul 20 18:04:02.758313 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.758290 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:04:02.875718 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.875691 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2zl7\" (UniqueName: \"kubernetes.io/projected/2d509dc5-d38f-4ce1-9dcf-aa724c96e301-kube-api-access-k2zl7\") pod \"2d509dc5-d38f-4ce1-9dcf-aa724c96e301\" (UID: \"2d509dc5-d38f-4ce1-9dcf-aa724c96e301\") " Jul 20 18:04:02.877951 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.877924 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d509dc5-d38f-4ce1-9dcf-aa724c96e301-kube-api-access-k2zl7" (OuterVolumeSpecName: "kube-api-access-k2zl7") pod "2d509dc5-d38f-4ce1-9dcf-aa724c96e301" (UID: "2d509dc5-d38f-4ce1-9dcf-aa724c96e301"). InnerVolumeSpecName "kube-api-access-k2zl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:04:02.977301 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:02.977267 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2zl7\" (UniqueName: \"kubernetes.io/projected/2d509dc5-d38f-4ce1-9dcf-aa724c96e301-kube-api-access-k2zl7\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:03.518557 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.518519 2567 generic.go:356] "Generic (PLEG): container finished" podID="2d509dc5-d38f-4ce1-9dcf-aa724c96e301" containerID="d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2" exitCode=0 Jul 20 18:04:03.519020 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.518561 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" Jul 20 18:04:03.519020 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.518610 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" event={"ID":"2d509dc5-d38f-4ce1-9dcf-aa724c96e301","Type":"ContainerDied","Data":"d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2"} Jul 20 18:04:03.519020 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.518659 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jpkj" event={"ID":"2d509dc5-d38f-4ce1-9dcf-aa724c96e301","Type":"ContainerDied","Data":"81ba2784b96fc80169e858b3d6467e0d81f1fab42a0d7f52a001243164cb86db"} Jul 20 18:04:03.519020 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.518676 2567 scope.go:117] "RemoveContainer" containerID="d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2" Jul 20 18:04:03.520331 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.520298 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" event={"ID":"1ccf85ab-fb69-4e69-ba1c-65d079d9086b","Type":"ContainerStarted","Data":"52d53fe394f28865cccf0847c3f733af87dfef6cef4317abe69413425559a077"} Jul 20 18:04:03.527832 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.527813 2567 scope.go:117] "RemoveContainer" containerID="d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2" Jul 20 18:04:03.528085 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:04:03.528065 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2\": container with ID starting with d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2 not found: ID does not exist" containerID="d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2" Jul 20 18:04:03.528148 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.528095 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2"} err="failed to get container status \"d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2\": rpc error: code = NotFound desc = could not find container \"d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2\": container with ID starting with d719949ae076af70b3d53f060e676bfac829186a065ad6e382a073549ed01cb2 not found: ID does not exist" Jul 20 18:04:03.536362 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.536341 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jpkj"] Jul 20 18:04:03.545518 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:03.545494 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jpkj"] Jul 20 18:04:05.440906 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:05.440871 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d509dc5-d38f-4ce1-9dcf-aa724c96e301" path="/var/lib/kubelet/pods/2d509dc5-d38f-4ce1-9dcf-aa724c96e301/volumes" Jul 20 18:04:10.004515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:10.004480 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:04:10.004925 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:10.004537 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:04:10.026996 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:10.026969 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:04:10.045652 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:10.045602 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" podStartSLOduration=8.399035044 podStartE2EDuration="11.045587624s" totalImagesPullingTime="2.64655258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:04:03.573143882 +0000 UTC m=+472.701261067" watchObservedRunningTime="2026-07-20 18:04:10.045587624 +0000 UTC m=+479.173704789" Jul 20 18:04:10.578203 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:10.578151 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lfw48" Jul 20 18:04:14.725509 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.725472 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z"] Jul 20 18:04:14.725889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.725848 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d509dc5-d38f-4ce1-9dcf-aa724c96e301" containerName="registry-server" Jul 20 18:04:14.725889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.725859 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d509dc5-d38f-4ce1-9dcf-aa724c96e301" containerName="registry-server" Jul 20 18:04:14.725958 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.725950 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d509dc5-d38f-4ce1-9dcf-aa724c96e301" containerName="registry-server" Jul 20 18:04:14.730968 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.730947 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.733596 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.733575 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5hqz2\"" Jul 20 18:04:14.736465 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.736441 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z"] Jul 20 18:04:14.887629 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.887592 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.887808 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.887695 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtgf\" (UniqueName: \"kubernetes.io/projected/d55eaab4-37b9-4792-9a21-8c3e77bf5710-kube-api-access-ngtgf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.887808 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.887725 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.988585 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.988490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtgf\" (UniqueName: \"kubernetes.io/projected/d55eaab4-37b9-4792-9a21-8c3e77bf5710-kube-api-access-ngtgf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.988585 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.988534 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.988585 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.988563 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.988987 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.988963 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.989046 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.988973 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:14.997320 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:14.997297 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtgf\" (UniqueName: \"kubernetes.io/projected/d55eaab4-37b9-4792-9a21-8c3e77bf5710-kube-api-access-ngtgf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:15.042119 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.042084 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:15.130159 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.130105 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx"] Jul 20 18:04:15.134650 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.134626 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.141862 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.141830 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx"] Jul 20 18:04:15.185554 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.185514 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z"] Jul 20 18:04:15.187844 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:15.187807 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd55eaab4_37b9_4792_9a21_8c3e77bf5710.slice/crio-1e4f0f0f0a9f73fa87e9acb3677b0374fd10172c74eb90f1a6c80ceeac72ffa8 WatchSource:0}: Error finding container 1e4f0f0f0a9f73fa87e9acb3677b0374fd10172c74eb90f1a6c80ceeac72ffa8: Status 404 returned error can't find the container with id 1e4f0f0f0a9f73fa87e9acb3677b0374fd10172c74eb90f1a6c80ceeac72ffa8 Jul 20 18:04:15.294594 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.294563 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhflg\" (UniqueName: \"kubernetes.io/projected/9decc64d-d9d2-41cc-a07f-3b407a61738f-kube-api-access-fhflg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.294749 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.294602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.294749 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.294644 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.395490 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.395449 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.395672 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.395552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhflg\" (UniqueName: \"kubernetes.io/projected/9decc64d-d9d2-41cc-a07f-3b407a61738f-kube-api-access-fhflg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.395672 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.395581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.395873 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.395852 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.395953 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.395931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.404595 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.404563 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhflg\" (UniqueName: \"kubernetes.io/projected/9decc64d-d9d2-41cc-a07f-3b407a61738f-kube-api-access-fhflg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.447654 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.447630 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:15.530525 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.530491 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw"] Jul 20 18:04:15.536129 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.536099 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.544375 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.544341 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw"] Jul 20 18:04:15.581514 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.581464 2567 generic.go:356] "Generic (PLEG): container finished" podID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerID="f300d1646467c458fd1b04d8af52e1ed4a76a912fa301fea312b377ddd8ad89b" exitCode=0 Jul 20 18:04:15.581686 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.581546 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" event={"ID":"d55eaab4-37b9-4792-9a21-8c3e77bf5710","Type":"ContainerDied","Data":"f300d1646467c458fd1b04d8af52e1ed4a76a912fa301fea312b377ddd8ad89b"} Jul 20 18:04:15.581686 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.581602 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" event={"ID":"d55eaab4-37b9-4792-9a21-8c3e77bf5710","Type":"ContainerStarted","Data":"1e4f0f0f0a9f73fa87e9acb3677b0374fd10172c74eb90f1a6c80ceeac72ffa8"} Jul 20 18:04:15.588906 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.588885 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx"] Jul 20 18:04:15.590251 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:15.590227 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9decc64d_d9d2_41cc_a07f_3b407a61738f.slice/crio-74028507abe1189277176182e179c72471bd56a214b8e9bcdbe5966b7c3a4234 WatchSource:0}: Error finding container 74028507abe1189277176182e179c72471bd56a214b8e9bcdbe5966b7c3a4234: Status 404 returned error can't find the container with id 74028507abe1189277176182e179c72471bd56a214b8e9bcdbe5966b7c3a4234 Jul 20 18:04:15.698625 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.698595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.698749 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.698643 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dg8\" (UniqueName: \"kubernetes.io/projected/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-kube-api-access-r9dg8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.698813 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.698789 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.800137 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.800099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.800658 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.800166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.800658 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.800236 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dg8\" (UniqueName: \"kubernetes.io/projected/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-kube-api-access-r9dg8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.800658 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.800524 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.800658 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.800552 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.809859 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.809829 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dg8\" (UniqueName: \"kubernetes.io/projected/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-kube-api-access-r9dg8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.849329 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.849301 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:15.928587 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.928553 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg"] Jul 20 18:04:15.933611 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.933379 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:15.944016 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.943981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg"] Jul 20 18:04:15.991364 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:15.991328 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw"] Jul 20 18:04:15.992751 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:15.992723 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a45a589_a43d_4b2a_8eb6_b69c460e6f6e.slice/crio-14a8ea7f9087fdd66c175f0940ea18a6bf83c237f545f552460db5113ca4d3dd WatchSource:0}: Error finding container 14a8ea7f9087fdd66c175f0940ea18a6bf83c237f545f552460db5113ca4d3dd: Status 404 returned error can't find the container with id 14a8ea7f9087fdd66c175f0940ea18a6bf83c237f545f552460db5113ca4d3dd Jul 20 18:04:16.103194 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.103141 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.103339 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.103231 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.103339 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.103257 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njst7\" (UniqueName: \"kubernetes.io/projected/0558c5bb-18b8-4534-9087-e203a0276770-kube-api-access-njst7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.204769 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.204733 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.204951 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.204804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.204951 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.204837 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njst7\" (UniqueName: \"kubernetes.io/projected/0558c5bb-18b8-4534-9087-e203a0276770-kube-api-access-njst7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.205108 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.205087 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.205166 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.205123 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.214697 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.214673 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njst7\" (UniqueName: \"kubernetes.io/projected/0558c5bb-18b8-4534-9087-e203a0276770-kube-api-access-njst7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.246322 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.246289 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:16.382156 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.382125 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg"] Jul 20 18:04:16.384036 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:16.384011 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0558c5bb_18b8_4534_9087_e203a0276770.slice/crio-69760c0909ef62f9c60d8cb1947d6a58d60f7cdec89e88dbf1631299f4806753 WatchSource:0}: Error finding container 69760c0909ef62f9c60d8cb1947d6a58d60f7cdec89e88dbf1631299f4806753: Status 404 returned error can't find the container with id 69760c0909ef62f9c60d8cb1947d6a58d60f7cdec89e88dbf1631299f4806753 Jul 20 18:04:16.590421 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.590339 2567 generic.go:356] "Generic (PLEG): container finished" podID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerID="582b34719fb7a1f40f2c1aecbe57215a4c406b00fb88fd499edc9db713c9d162" exitCode=0 Jul 20 18:04:16.590584 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.590414 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" event={"ID":"9decc64d-d9d2-41cc-a07f-3b407a61738f","Type":"ContainerDied","Data":"582b34719fb7a1f40f2c1aecbe57215a4c406b00fb88fd499edc9db713c9d162"} Jul 20 18:04:16.590584 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.590462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" event={"ID":"9decc64d-d9d2-41cc-a07f-3b407a61738f","Type":"ContainerStarted","Data":"74028507abe1189277176182e179c72471bd56a214b8e9bcdbe5966b7c3a4234"} Jul 20 18:04:16.592327 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.592299 2567 generic.go:356] "Generic (PLEG): container finished" podID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerID="33c09044710c960a74b0202d2b6585ec9ad309f0d6b9f8df70cd6c5c8cade2e3" exitCode=0 Jul 20 18:04:16.592441 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.592373 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" event={"ID":"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e","Type":"ContainerDied","Data":"33c09044710c960a74b0202d2b6585ec9ad309f0d6b9f8df70cd6c5c8cade2e3"} Jul 20 18:04:16.592441 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.592423 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" event={"ID":"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e","Type":"ContainerStarted","Data":"14a8ea7f9087fdd66c175f0940ea18a6bf83c237f545f552460db5113ca4d3dd"} Jul 20 18:04:16.593953 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.593935 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" event={"ID":"0558c5bb-18b8-4534-9087-e203a0276770","Type":"ContainerStarted","Data":"514379c93513c41ce767ec9d93f0ab552760dd99efbaa54196b6662e0c08bc59"} Jul 20 18:04:16.594068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:16.593973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" event={"ID":"0558c5bb-18b8-4534-9087-e203a0276770","Type":"ContainerStarted","Data":"69760c0909ef62f9c60d8cb1947d6a58d60f7cdec89e88dbf1631299f4806753"} Jul 20 18:04:17.601115 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:17.601079 2567 generic.go:356] "Generic (PLEG): container finished" podID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerID="3b4a5d580a05a506d9ae3c701c6483b1b9bb9921c9b67687541891daacbbecbe" exitCode=0 Jul 20 18:04:17.601572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:17.601172 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" event={"ID":"d55eaab4-37b9-4792-9a21-8c3e77bf5710","Type":"ContainerDied","Data":"3b4a5d580a05a506d9ae3c701c6483b1b9bb9921c9b67687541891daacbbecbe"} Jul 20 18:04:17.602858 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:17.602742 2567 generic.go:356] "Generic (PLEG): container finished" podID="0558c5bb-18b8-4534-9087-e203a0276770" containerID="514379c93513c41ce767ec9d93f0ab552760dd99efbaa54196b6662e0c08bc59" exitCode=0 Jul 20 18:04:17.602858 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:17.602771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" event={"ID":"0558c5bb-18b8-4534-9087-e203a0276770","Type":"ContainerDied","Data":"514379c93513c41ce767ec9d93f0ab552760dd99efbaa54196b6662e0c08bc59"} Jul 20 18:04:18.611294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:18.611262 2567 generic.go:356] "Generic (PLEG): container finished" podID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerID="90ae3eaa0a2d6816d73531a2345b477bf7136d4299208fe149492882dfd47ed1" exitCode=0 Jul 20 18:04:18.611734 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:18.611346 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" event={"ID":"d55eaab4-37b9-4792-9a21-8c3e77bf5710","Type":"ContainerDied","Data":"90ae3eaa0a2d6816d73531a2345b477bf7136d4299208fe149492882dfd47ed1"} Jul 20 18:04:18.613131 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:18.613101 2567 generic.go:356] "Generic (PLEG): container finished" podID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerID="4d38663460833aa3bbbbd5dc9c0bbe82d897ff5cecd2672d271ca93f23aec725" exitCode=0 Jul 20 18:04:18.613267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:18.613199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" event={"ID":"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e","Type":"ContainerDied","Data":"4d38663460833aa3bbbbd5dc9c0bbe82d897ff5cecd2672d271ca93f23aec725"} Jul 20 18:04:18.615144 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:18.615122 2567 generic.go:356] "Generic (PLEG): container finished" podID="0558c5bb-18b8-4534-9087-e203a0276770" containerID="b46473551cab7929f0291403ed951463b2b2ad77a4b1acf62becb6feb5de0481" exitCode=0 Jul 20 18:04:18.615246 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:18.615175 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" event={"ID":"0558c5bb-18b8-4534-9087-e203a0276770","Type":"ContainerDied","Data":"b46473551cab7929f0291403ed951463b2b2ad77a4b1acf62becb6feb5de0481"} Jul 20 18:04:19.622797 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.622740 2567 generic.go:356] "Generic (PLEG): container finished" podID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerID="287305b9a0f7a4f0087fa44d92136d441cc731d696a80fb2b5a24553cb71b585" exitCode=0 Jul 20 18:04:19.623160 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.622816 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" event={"ID":"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e","Type":"ContainerDied","Data":"287305b9a0f7a4f0087fa44d92136d441cc731d696a80fb2b5a24553cb71b585"} Jul 20 18:04:19.625084 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.625058 2567 generic.go:356] "Generic (PLEG): container finished" podID="0558c5bb-18b8-4534-9087-e203a0276770" containerID="4b29e5c8992cc392cec4a2410ffdf929725a3e63d685b4154ed2064cd958c1b7" exitCode=0 Jul 20 18:04:19.625200 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.625085 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" event={"ID":"0558c5bb-18b8-4534-9087-e203a0276770","Type":"ContainerDied","Data":"4b29e5c8992cc392cec4a2410ffdf929725a3e63d685b4154ed2064cd958c1b7"} Jul 20 18:04:19.627076 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.627055 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" event={"ID":"9decc64d-d9d2-41cc-a07f-3b407a61738f","Type":"ContainerStarted","Data":"70e62016f14706c323cc6efa6b3e81a521a9dc077014a6ee1fc04652b2ae4dbe"} Jul 20 18:04:19.761172 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.761149 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:19.938454 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.938423 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngtgf\" (UniqueName: \"kubernetes.io/projected/d55eaab4-37b9-4792-9a21-8c3e77bf5710-kube-api-access-ngtgf\") pod \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " Jul 20 18:04:19.938614 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.938539 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-util\") pod \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " Jul 20 18:04:19.938614 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.938562 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-bundle\") pod \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\" (UID: \"d55eaab4-37b9-4792-9a21-8c3e77bf5710\") " Jul 20 18:04:19.939113 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.939082 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-bundle" (OuterVolumeSpecName: "bundle") pod "d55eaab4-37b9-4792-9a21-8c3e77bf5710" (UID: "d55eaab4-37b9-4792-9a21-8c3e77bf5710"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:19.940463 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.940438 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d55eaab4-37b9-4792-9a21-8c3e77bf5710-kube-api-access-ngtgf" (OuterVolumeSpecName: "kube-api-access-ngtgf") pod "d55eaab4-37b9-4792-9a21-8c3e77bf5710" (UID: "d55eaab4-37b9-4792-9a21-8c3e77bf5710"). InnerVolumeSpecName "kube-api-access-ngtgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:04:19.944330 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:19.944309 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-util" (OuterVolumeSpecName: "util") pod "d55eaab4-37b9-4792-9a21-8c3e77bf5710" (UID: "d55eaab4-37b9-4792-9a21-8c3e77bf5710"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:20.039549 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.039517 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngtgf\" (UniqueName: \"kubernetes.io/projected/d55eaab4-37b9-4792-9a21-8c3e77bf5710-kube-api-access-ngtgf\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:20.039549 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.039546 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:20.039549 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.039557 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d55eaab4-37b9-4792-9a21-8c3e77bf5710-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:20.643343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.643309 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" Jul 20 18:04:20.643343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.643330 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z" event={"ID":"d55eaab4-37b9-4792-9a21-8c3e77bf5710","Type":"ContainerDied","Data":"1e4f0f0f0a9f73fa87e9acb3677b0374fd10172c74eb90f1a6c80ceeac72ffa8"} Jul 20 18:04:20.643878 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.643417 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e4f0f0f0a9f73fa87e9acb3677b0374fd10172c74eb90f1a6c80ceeac72ffa8" Jul 20 18:04:20.645913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.645879 2567 generic.go:356] "Generic (PLEG): container finished" podID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerID="70e62016f14706c323cc6efa6b3e81a521a9dc077014a6ee1fc04652b2ae4dbe" exitCode=0 Jul 20 18:04:20.646050 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.645917 2567 generic.go:356] "Generic (PLEG): container finished" podID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerID="5fa54d86f90165961b4a269c1f6fecc6d9e4114462dd88b487404e854ccaa2f4" exitCode=0 Jul 20 18:04:20.646050 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.645963 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" event={"ID":"9decc64d-d9d2-41cc-a07f-3b407a61738f","Type":"ContainerDied","Data":"70e62016f14706c323cc6efa6b3e81a521a9dc077014a6ee1fc04652b2ae4dbe"} Jul 20 18:04:20.646050 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.646027 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" event={"ID":"9decc64d-d9d2-41cc-a07f-3b407a61738f","Type":"ContainerDied","Data":"5fa54d86f90165961b4a269c1f6fecc6d9e4114462dd88b487404e854ccaa2f4"} Jul 20 18:04:20.796821 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.796797 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:20.798743 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.798724 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:20.946470 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.946385 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-util\") pod \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " Jul 20 18:04:20.946470 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.946433 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9dg8\" (UniqueName: \"kubernetes.io/projected/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-kube-api-access-r9dg8\") pod \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " Jul 20 18:04:20.946679 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.946475 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-util\") pod \"0558c5bb-18b8-4534-9087-e203a0276770\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " Jul 20 18:04:20.946679 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.946510 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-bundle\") pod \"0558c5bb-18b8-4534-9087-e203a0276770\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " Jul 20 18:04:20.946679 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.946572 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njst7\" (UniqueName: \"kubernetes.io/projected/0558c5bb-18b8-4534-9087-e203a0276770-kube-api-access-njst7\") pod \"0558c5bb-18b8-4534-9087-e203a0276770\" (UID: \"0558c5bb-18b8-4534-9087-e203a0276770\") " Jul 20 18:04:20.946679 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.946602 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-bundle\") pod \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\" (UID: \"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e\") " Jul 20 18:04:20.947071 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.947037 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-bundle" (OuterVolumeSpecName: "bundle") pod "0558c5bb-18b8-4534-9087-e203a0276770" (UID: "0558c5bb-18b8-4534-9087-e203a0276770"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:20.947413 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.947384 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-bundle" (OuterVolumeSpecName: "bundle") pod "4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" (UID: "4a45a589-a43d-4b2a-8eb6-b69c460e6f6e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:20.948786 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.948758 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-kube-api-access-r9dg8" (OuterVolumeSpecName: "kube-api-access-r9dg8") pod "4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" (UID: "4a45a589-a43d-4b2a-8eb6-b69c460e6f6e"). InnerVolumeSpecName "kube-api-access-r9dg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:04:20.948866 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.948799 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0558c5bb-18b8-4534-9087-e203a0276770-kube-api-access-njst7" (OuterVolumeSpecName: "kube-api-access-njst7") pod "0558c5bb-18b8-4534-9087-e203a0276770" (UID: "0558c5bb-18b8-4534-9087-e203a0276770"). InnerVolumeSpecName "kube-api-access-njst7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:04:20.952450 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.952419 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-util" (OuterVolumeSpecName: "util") pod "4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" (UID: "4a45a589-a43d-4b2a-8eb6-b69c460e6f6e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:20.954721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:20.954698 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-util" (OuterVolumeSpecName: "util") pod "0558c5bb-18b8-4534-9087-e203a0276770" (UID: "0558c5bb-18b8-4534-9087-e203a0276770"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:21.047932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.047895 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njst7\" (UniqueName: \"kubernetes.io/projected/0558c5bb-18b8-4534-9087-e203a0276770-kube-api-access-njst7\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.047932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.047927 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.047932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.047939 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.048169 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.047947 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9dg8\" (UniqueName: \"kubernetes.io/projected/4a45a589-a43d-4b2a-8eb6-b69c460e6f6e-kube-api-access-r9dg8\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.048169 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.047956 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.048169 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.047965 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0558c5bb-18b8-4534-9087-e203a0276770-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.653074 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.653045 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" Jul 20 18:04:21.653074 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.653051 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw" event={"ID":"4a45a589-a43d-4b2a-8eb6-b69c460e6f6e","Type":"ContainerDied","Data":"14a8ea7f9087fdd66c175f0940ea18a6bf83c237f545f552460db5113ca4d3dd"} Jul 20 18:04:21.653629 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.653094 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14a8ea7f9087fdd66c175f0940ea18a6bf83c237f545f552460db5113ca4d3dd" Jul 20 18:04:21.655287 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.655261 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" Jul 20 18:04:21.655406 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.655280 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg" event={"ID":"0558c5bb-18b8-4534-9087-e203a0276770","Type":"ContainerDied","Data":"69760c0909ef62f9c60d8cb1947d6a58d60f7cdec89e88dbf1631299f4806753"} Jul 20 18:04:21.655406 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.655327 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69760c0909ef62f9c60d8cb1947d6a58d60f7cdec89e88dbf1631299f4806753" Jul 20 18:04:21.791563 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.791523 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:21.854110 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.854080 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-util\") pod \"9decc64d-d9d2-41cc-a07f-3b407a61738f\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " Jul 20 18:04:21.854269 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.854147 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhflg\" (UniqueName: \"kubernetes.io/projected/9decc64d-d9d2-41cc-a07f-3b407a61738f-kube-api-access-fhflg\") pod \"9decc64d-d9d2-41cc-a07f-3b407a61738f\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " Jul 20 18:04:21.854269 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.854171 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-bundle\") pod \"9decc64d-d9d2-41cc-a07f-3b407a61738f\" (UID: \"9decc64d-d9d2-41cc-a07f-3b407a61738f\") " Jul 20 18:04:21.854683 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.854662 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-bundle" (OuterVolumeSpecName: "bundle") pod "9decc64d-d9d2-41cc-a07f-3b407a61738f" (UID: "9decc64d-d9d2-41cc-a07f-3b407a61738f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:21.856284 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.856253 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9decc64d-d9d2-41cc-a07f-3b407a61738f-kube-api-access-fhflg" (OuterVolumeSpecName: "kube-api-access-fhflg") pod "9decc64d-d9d2-41cc-a07f-3b407a61738f" (UID: "9decc64d-d9d2-41cc-a07f-3b407a61738f"). InnerVolumeSpecName "kube-api-access-fhflg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:04:21.861122 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.861091 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-util" (OuterVolumeSpecName: "util") pod "9decc64d-d9d2-41cc-a07f-3b407a61738f" (UID: "9decc64d-d9d2-41cc-a07f-3b407a61738f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:04:21.955264 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.955227 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-util\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.955264 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.955258 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhflg\" (UniqueName: \"kubernetes.io/projected/9decc64d-d9d2-41cc-a07f-3b407a61738f-kube-api-access-fhflg\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:21.955264 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:21.955268 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9decc64d-d9d2-41cc-a07f-3b407a61738f-bundle\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:04:22.662276 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:22.662243 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" Jul 20 18:04:22.662276 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:22.662265 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx" event={"ID":"9decc64d-d9d2-41cc-a07f-3b407a61738f","Type":"ContainerDied","Data":"74028507abe1189277176182e179c72471bd56a214b8e9bcdbe5966b7c3a4234"} Jul 20 18:04:22.662725 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:22.662308 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74028507abe1189277176182e179c72471bd56a214b8e9bcdbe5966b7c3a4234" Jul 20 18:04:22.683966 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:04:22.683943 2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9decc64d_d9d2_41cc_a07f_3b407a61738f.slice\": RecentStats: unable to find data in memory cache]" Jul 20 18:04:30.631158 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.631117 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5"] Jul 20 18:04:30.632021 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.631996 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="util" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632036 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="util" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632048 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="util" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632057 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="util" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632068 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="pull" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632076 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="pull" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632094 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="extract" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632102 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="extract" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632122 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="pull" Jul 20 18:04:30.632135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632136 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="pull" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632155 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632164 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632199 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="util" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632208 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="util" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632227 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632235 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632260 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632269 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632279 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="pull" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632288 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="pull" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632307 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="util" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632315 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="util" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632335 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="pull" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632342 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="pull" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632517 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a45a589-a43d-4b2a-8eb6-b69c460e6f6e" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632539 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9decc64d-d9d2-41cc-a07f-3b407a61738f" containerName="extract" Jul 20 18:04:30.632541 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632550 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d55eaab4-37b9-4792-9a21-8c3e77bf5710" containerName="extract" Jul 20 18:04:30.633253 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.632568 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0558c5bb-18b8-4534-9087-e203a0276770" containerName="extract" Jul 20 18:04:30.636903 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.636360 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:30.641874 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.641852 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-b6dpr\"" Jul 20 18:04:30.644809 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.644741 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5"] Jul 20 18:04:30.735702 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.735669 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmgkj\" (UniqueName: \"kubernetes.io/projected/998834b8-d933-4398-a69b-c0e6bac7348f-kube-api-access-pmgkj\") pod \"limitador-operator-controller-manager-859d7b78bd-6blx5\" (UID: \"998834b8-d933-4398-a69b-c0e6bac7348f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:30.836702 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.836647 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmgkj\" (UniqueName: \"kubernetes.io/projected/998834b8-d933-4398-a69b-c0e6bac7348f-kube-api-access-pmgkj\") pod \"limitador-operator-controller-manager-859d7b78bd-6blx5\" (UID: \"998834b8-d933-4398-a69b-c0e6bac7348f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:30.845294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.845264 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmgkj\" (UniqueName: \"kubernetes.io/projected/998834b8-d933-4398-a69b-c0e6bac7348f-kube-api-access-pmgkj\") pod \"limitador-operator-controller-manager-859d7b78bd-6blx5\" (UID: \"998834b8-d933-4398-a69b-c0e6bac7348f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:30.948433 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:30.948345 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:31.086268 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:31.086243 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5"] Jul 20 18:04:31.087877 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:31.087848 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998834b8_d933_4398_a69b_c0e6bac7348f.slice/crio-7ee6392ea9deca5d6802ec2bc7ff547da721f893a13201ac341304113d5b1fe3 WatchSource:0}: Error finding container 7ee6392ea9deca5d6802ec2bc7ff547da721f893a13201ac341304113d5b1fe3: Status 404 returned error can't find the container with id 7ee6392ea9deca5d6802ec2bc7ff547da721f893a13201ac341304113d5b1fe3 Jul 20 18:04:31.713749 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:31.713710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" event={"ID":"998834b8-d933-4398-a69b-c0e6bac7348f","Type":"ContainerStarted","Data":"7ee6392ea9deca5d6802ec2bc7ff547da721f893a13201ac341304113d5b1fe3"} Jul 20 18:04:33.726101 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:33.726062 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" event={"ID":"998834b8-d933-4398-a69b-c0e6bac7348f","Type":"ContainerStarted","Data":"6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95"} Jul 20 18:04:33.726510 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:33.726205 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:33.752479 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:33.752431 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" podStartSLOduration=2.127763485 podStartE2EDuration="3.752413907s" totalImagesPullingTime="1.624650422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:04:33.750838247 +0000 UTC m=+502.878955414" watchObservedRunningTime="2026-07-20 18:04:33.752413907 +0000 UTC m=+502.880531073" Jul 20 18:04:34.043231 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.043134 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh"] Jul 20 18:04:34.046416 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.046397 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:34.049274 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.049254 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wqwg9\"" Jul 20 18:04:34.049369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.049324 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 18:04:34.065031 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.065006 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh"] Jul 20 18:04:34.168594 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.168562 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brr2h\" (UniqueName: \"kubernetes.io/projected/945777a1-35fb-4ae0-83f9-90584f9adaf7-kube-api-access-brr2h\") pod \"dns-operator-controller-manager-864dd78c4c-lq9vh\" (UID: \"945777a1-35fb-4ae0-83f9-90584f9adaf7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:34.269753 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.269719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brr2h\" (UniqueName: \"kubernetes.io/projected/945777a1-35fb-4ae0-83f9-90584f9adaf7-kube-api-access-brr2h\") pod \"dns-operator-controller-manager-864dd78c4c-lq9vh\" (UID: \"945777a1-35fb-4ae0-83f9-90584f9adaf7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:34.278255 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.278233 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brr2h\" (UniqueName: \"kubernetes.io/projected/945777a1-35fb-4ae0-83f9-90584f9adaf7-kube-api-access-brr2h\") pod \"dns-operator-controller-manager-864dd78c4c-lq9vh\" (UID: \"945777a1-35fb-4ae0-83f9-90584f9adaf7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:34.356735 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.356702 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:34.495441 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.493174 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh"] Jul 20 18:04:34.495653 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:34.495627 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945777a1_35fb_4ae0_83f9_90584f9adaf7.slice/crio-7eb2498a74bc313110f9cf2c872e3901363e9563ea3f555f5ffa042c829fbcc1 WatchSource:0}: Error finding container 7eb2498a74bc313110f9cf2c872e3901363e9563ea3f555f5ffa042c829fbcc1: Status 404 returned error can't find the container with id 7eb2498a74bc313110f9cf2c872e3901363e9563ea3f555f5ffa042c829fbcc1 Jul 20 18:04:34.732762 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:34.732677 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" event={"ID":"945777a1-35fb-4ae0-83f9-90584f9adaf7","Type":"ContainerStarted","Data":"7eb2498a74bc313110f9cf2c872e3901363e9563ea3f555f5ffa042c829fbcc1"} Jul 20 18:04:36.746355 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:36.746305 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" event={"ID":"945777a1-35fb-4ae0-83f9-90584f9adaf7","Type":"ContainerStarted","Data":"92dae1b46eecda8441da3c26054038cc4e1168935860779bcc24f364359b2480"} Jul 20 18:04:36.746740 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:36.746384 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:36.765156 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:36.765108 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" podStartSLOduration=0.729333233 podStartE2EDuration="2.765093181s" totalImagesPullingTime="2.035759948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:04:36.764401902 +0000 UTC m=+505.892519105" watchObservedRunningTime="2026-07-20 18:04:36.765093181 +0000 UTC m=+505.893210348" Jul 20 18:04:37.841791 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:37.841759 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z"] Jul 20 18:04:37.844494 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:37.844476 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:04:37.847484 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:37.847464 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-t6r7l\"" Jul 20 18:04:37.865003 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:37.864973 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z"] Jul 20 18:04:37.908091 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:37.908049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4r8l\" (UniqueName: \"kubernetes.io/projected/b2e119da-1f94-4777-9107-2f152558ab06-kube-api-access-l4r8l\") pod \"authorino-operator-75cc8d5cb7-kmq6z\" (UID: \"b2e119da-1f94-4777-9107-2f152558ab06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:04:38.008572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:38.008535 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4r8l\" (UniqueName: \"kubernetes.io/projected/b2e119da-1f94-4777-9107-2f152558ab06-kube-api-access-l4r8l\") pod \"authorino-operator-75cc8d5cb7-kmq6z\" (UID: \"b2e119da-1f94-4777-9107-2f152558ab06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:04:38.025159 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:38.025130 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4r8l\" (UniqueName: \"kubernetes.io/projected/b2e119da-1f94-4777-9107-2f152558ab06-kube-api-access-l4r8l\") pod \"authorino-operator-75cc8d5cb7-kmq6z\" (UID: \"b2e119da-1f94-4777-9107-2f152558ab06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:04:38.154254 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:38.154143 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:04:38.302517 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:38.302478 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z"] Jul 20 18:04:38.303265 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:38.303235 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e119da_1f94_4777_9107_2f152558ab06.slice/crio-f5037902fe4c9a527545c93f55c60e789df737bdeb99aed29e7fb5eae9f13a6f WatchSource:0}: Error finding container f5037902fe4c9a527545c93f55c60e789df737bdeb99aed29e7fb5eae9f13a6f: Status 404 returned error can't find the container with id f5037902fe4c9a527545c93f55c60e789df737bdeb99aed29e7fb5eae9f13a6f Jul 20 18:04:38.759062 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:38.759031 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" event={"ID":"b2e119da-1f94-4777-9107-2f152558ab06","Type":"ContainerStarted","Data":"f5037902fe4c9a527545c93f55c60e789df737bdeb99aed29e7fb5eae9f13a6f"} Jul 20 18:04:41.784758 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:41.784716 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" event={"ID":"b2e119da-1f94-4777-9107-2f152558ab06","Type":"ContainerStarted","Data":"fd80d1eae022645e4abfc7be47bdce7803698709bb2ff0855b305fc9e8b177ed"} Jul 20 18:04:41.785164 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:41.784845 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:04:41.804771 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:41.804714 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" podStartSLOduration=2.091569629 podStartE2EDuration="4.804699428s" totalImagesPullingTime="2.713129799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:04:41.803548755 +0000 UTC m=+510.931665922" watchObservedRunningTime="2026-07-20 18:04:41.804699428 +0000 UTC m=+510.932816594" Jul 20 18:04:43.132320 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.132283 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk"] Jul 20 18:04:43.134726 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.134709 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.137354 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.137334 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fn4h6\"" Jul 20 18:04:43.151468 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.151442 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk"] Jul 20 18:04:43.257924 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.257889 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc685221-7f49-4759-9c6c-dc02ec4c8203-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.258070 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.257938 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tmzc\" (UniqueName: \"kubernetes.io/projected/dc685221-7f49-4759-9c6c-dc02ec4c8203-kube-api-access-5tmzc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.359487 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.359445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc685221-7f49-4759-9c6c-dc02ec4c8203-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.359487 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.359493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tmzc\" (UniqueName: \"kubernetes.io/projected/dc685221-7f49-4759-9c6c-dc02ec4c8203-kube-api-access-5tmzc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.359868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.359848 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc685221-7f49-4759-9c6c-dc02ec4c8203-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.383484 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.383410 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tmzc\" (UniqueName: \"kubernetes.io/projected/dc685221-7f49-4759-9c6c-dc02ec4c8203-kube-api-access-5tmzc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.445697 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.445667 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:43.587897 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.587869 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk"] Jul 20 18:04:43.590577 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:04:43.590549 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc685221_7f49_4759_9c6c_dc02ec4c8203.slice/crio-70a72ded5735906dc8666b1717ef91132378ae6f74d61be0ae99331d5d55b712 WatchSource:0}: Error finding container 70a72ded5735906dc8666b1717ef91132378ae6f74d61be0ae99331d5d55b712: Status 404 returned error can't find the container with id 70a72ded5735906dc8666b1717ef91132378ae6f74d61be0ae99331d5d55b712 Jul 20 18:04:43.796579 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:43.796549 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" event={"ID":"dc685221-7f49-4759-9c6c-dc02ec4c8203","Type":"ContainerStarted","Data":"70a72ded5735906dc8666b1717ef91132378ae6f74d61be0ae99331d5d55b712"} Jul 20 18:04:44.735053 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:44.735016 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:04:47.754676 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:47.754644 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lq9vh" Jul 20 18:04:50.843249 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:50.843143 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" event={"ID":"dc685221-7f49-4759-9c6c-dc02ec4c8203","Type":"ContainerStarted","Data":"0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86"} Jul 20 18:04:50.843634 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:50.843257 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:04:50.870878 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:50.870828 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" podStartSLOduration=0.905870711 podStartE2EDuration="7.870811952s" totalImagesPullingTime="6.964941241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:04:50.868003604 +0000 UTC m=+519.996120770" watchObservedRunningTime="2026-07-20 18:04:50.870811952 +0000 UTC m=+519.998929161" Jul 20 18:04:52.792258 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:04:52.792228 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kmq6z" Jul 20 18:05:01.852275 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:01.852245 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:03.853797 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.853715 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk"] Jul 20 18:05:03.854284 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.853934 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" containerName="manager" containerID="cri-o://0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86" gracePeriod=2 Jul 20 18:05:03.873722 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.873693 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk"] Jul 20 18:05:03.884976 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:03.884930 2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"dc685221-7f49-4759-9c6c-dc02ec4c8203\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\": pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:03.893155 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.893128 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp"] Jul 20 18:05:03.893607 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.893591 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" containerName="manager" Jul 20 18:05:03.893697 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.893609 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" containerName="manager" Jul 20 18:05:03.893755 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.893702 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" containerName="manager" Jul 20 18:05:03.896016 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.895997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:03.899415 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:03.899387 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:03.915130 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.915102 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp"] Jul 20 18:05:03.939983 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.939946 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5"] Jul 20 18:05:03.940343 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.940292 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" containerName="manager" containerID="cri-o://6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95" gracePeriod=2 Jul 20 18:05:03.945290 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.945263 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5"] Jul 20 18:05:03.945739 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:03.945710 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:03.948614 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:03.948584 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:03.951959 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.951934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cef4bdbf-1cce-4b99-afbe-768d643bf97e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bljp\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:03.952060 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.951990 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxgwt\" (UniqueName: \"kubernetes.io/projected/cef4bdbf-1cce-4b99-afbe-768d643bf97e-kube-api-access-qxgwt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bljp\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:03.983723 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.983674 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh"] Jul 20 18:05:03.984332 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.984310 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" containerName="manager" Jul 20 18:05:03.984656 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.984335 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" containerName="manager" Jul 20 18:05:03.984656 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.984475 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" containerName="manager" Jul 20 18:05:03.987357 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:03.987325 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:03.990247 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:03.990213 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:03.992531 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:03.992492 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.005998 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.005874 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh"] Jul 20 18:05:04.053578 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.053550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cef4bdbf-1cce-4b99-afbe-768d643bf97e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bljp\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:04.053744 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.053593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxgwt\" (UniqueName: \"kubernetes.io/projected/cef4bdbf-1cce-4b99-afbe-768d643bf97e-kube-api-access-qxgwt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bljp\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:04.053744 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.053654 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf52m\" (UniqueName: \"kubernetes.io/projected/99a3c160-632a-4a1b-8f76-c20917e9c112-kube-api-access-wf52m\") pod \"limitador-operator-controller-manager-859d7b78bd-4j4jh\" (UID: \"99a3c160-632a-4a1b-8f76-c20917e9c112\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:04.053991 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.053970 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cef4bdbf-1cce-4b99-afbe-768d643bf97e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bljp\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:04.066691 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.066667 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxgwt\" (UniqueName: \"kubernetes.io/projected/cef4bdbf-1cce-4b99-afbe-768d643bf97e-kube-api-access-qxgwt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7bljp\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:04.113974 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.113905 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.116443 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.116411 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.119064 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.119040 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.154582 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.154550 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc685221-7f49-4759-9c6c-dc02ec4c8203-extensions-socket-volume\") pod \"dc685221-7f49-4759-9c6c-dc02ec4c8203\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " Jul 20 18:05:04.154763 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.154639 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tmzc\" (UniqueName: \"kubernetes.io/projected/dc685221-7f49-4759-9c6c-dc02ec4c8203-kube-api-access-5tmzc\") pod \"dc685221-7f49-4759-9c6c-dc02ec4c8203\" (UID: \"dc685221-7f49-4759-9c6c-dc02ec4c8203\") " Jul 20 18:05:04.154882 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.154863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf52m\" (UniqueName: \"kubernetes.io/projected/99a3c160-632a-4a1b-8f76-c20917e9c112-kube-api-access-wf52m\") pod \"limitador-operator-controller-manager-859d7b78bd-4j4jh\" (UID: \"99a3c160-632a-4a1b-8f76-c20917e9c112\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:04.155059 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.155034 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc685221-7f49-4759-9c6c-dc02ec4c8203-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc685221-7f49-4759-9c6c-dc02ec4c8203" (UID: "dc685221-7f49-4759-9c6c-dc02ec4c8203"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:05:04.156764 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.156733 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc685221-7f49-4759-9c6c-dc02ec4c8203-kube-api-access-5tmzc" (OuterVolumeSpecName: "kube-api-access-5tmzc") pod "dc685221-7f49-4759-9c6c-dc02ec4c8203" (UID: "dc685221-7f49-4759-9c6c-dc02ec4c8203"). InnerVolumeSpecName "kube-api-access-5tmzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:05:04.165326 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.165298 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf52m\" (UniqueName: \"kubernetes.io/projected/99a3c160-632a-4a1b-8f76-c20917e9c112-kube-api-access-wf52m\") pod \"limitador-operator-controller-manager-859d7b78bd-4j4jh\" (UID: \"99a3c160-632a-4a1b-8f76-c20917e9c112\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:04.214986 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.214961 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.219757 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.219731 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.222913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.222894 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:04.224014 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.223996 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.255321 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.255275 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmgkj\" (UniqueName: \"kubernetes.io/projected/998834b8-d933-4398-a69b-c0e6bac7348f-kube-api-access-pmgkj\") pod \"998834b8-d933-4398-a69b-c0e6bac7348f\" (UID: \"998834b8-d933-4398-a69b-c0e6bac7348f\") " Jul 20 18:05:04.255528 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.255469 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc685221-7f49-4759-9c6c-dc02ec4c8203-extensions-socket-volume\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:04.255528 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.255481 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tmzc\" (UniqueName: \"kubernetes.io/projected/dc685221-7f49-4759-9c6c-dc02ec4c8203-kube-api-access-5tmzc\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:04.257278 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.257246 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/998834b8-d933-4398-a69b-c0e6bac7348f-kube-api-access-pmgkj" (OuterVolumeSpecName: "kube-api-access-pmgkj") pod "998834b8-d933-4398-a69b-c0e6bac7348f" (UID: "998834b8-d933-4398-a69b-c0e6bac7348f"). InnerVolumeSpecName "kube-api-access-pmgkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:05:04.306408 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.306377 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:04.356499 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.356464 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmgkj\" (UniqueName: \"kubernetes.io/projected/998834b8-d933-4398-a69b-c0e6bac7348f-kube-api-access-pmgkj\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:04.388003 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.387976 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp"] Jul 20 18:05:04.389998 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:05:04.389970 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef4bdbf_1cce_4b99_afbe_768d643bf97e.slice/crio-04e78d6650486bdf2d046706a24bc94a895d31f18b927e808a847020cac041c7 WatchSource:0}: Error finding container 04e78d6650486bdf2d046706a24bc94a895d31f18b927e808a847020cac041c7: Status 404 returned error can't find the container with id 04e78d6650486bdf2d046706a24bc94a895d31f18b927e808a847020cac041c7 Jul 20 18:05:04.457491 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.457453 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh"] Jul 20 18:05:04.459725 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:05:04.459699 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a3c160_632a_4a1b_8f76_c20917e9c112.slice/crio-5c0809d527d582cdcd34e6a764469ebf8bb7af74035e766af077f77982e2db6d WatchSource:0}: Error finding container 5c0809d527d582cdcd34e6a764469ebf8bb7af74035e766af077f77982e2db6d: Status 404 returned error can't find the container with id 5c0809d527d582cdcd34e6a764469ebf8bb7af74035e766af077f77982e2db6d Jul 20 18:05:04.934517 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.934482 2567 generic.go:356] "Generic (PLEG): container finished" podID="998834b8-d933-4398-a69b-c0e6bac7348f" containerID="6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95" exitCode=0 Jul 20 18:05:04.934970 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.934526 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.934970 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.934580 2567 scope.go:117] "RemoveContainer" containerID="6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95" Jul 20 18:05:04.936383 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.936352 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" event={"ID":"cef4bdbf-1cce-4b99-afbe-768d643bf97e","Type":"ContainerStarted","Data":"0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4"} Jul 20 18:05:04.936521 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.936416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" event={"ID":"cef4bdbf-1cce-4b99-afbe-768d643bf97e","Type":"ContainerStarted","Data":"04e78d6650486bdf2d046706a24bc94a895d31f18b927e808a847020cac041c7"} Jul 20 18:05:04.936521 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.936449 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:04.937807 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.937774 2567 generic.go:356] "Generic (PLEG): container finished" podID="dc685221-7f49-4759-9c6c-dc02ec4c8203" containerID="0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86" exitCode=0 Jul 20 18:05:04.937909 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.937812 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.938651 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.938607 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.939854 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.939829 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" event={"ID":"99a3c160-632a-4a1b-8f76-c20917e9c112","Type":"ContainerStarted","Data":"2f54fd0dfba1fe62ff4cf274a8221dfa99d013b66ac3fb7f424e55d37c366c40"} Jul 20 18:05:04.939960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.939880 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" event={"ID":"99a3c160-632a-4a1b-8f76-c20917e9c112","Type":"ContainerStarted","Data":"5c0809d527d582cdcd34e6a764469ebf8bb7af74035e766af077f77982e2db6d"} Jul 20 18:05:04.940250 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.940233 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:04.940549 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.940525 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.942598 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.942578 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.944412 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.944381 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:04.946501 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.946479 2567 scope.go:117] "RemoveContainer" containerID="6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95" Jul 20 18:05:04.946806 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.946780 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95\": container with ID starting with 6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95 not found: ID does not exist" containerID="6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95" Jul 20 18:05:04.946896 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.946819 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95"} err="failed to get container status \"6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95\": rpc error: code = NotFound desc = could not find container \"6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95\": container with ID starting with 6badb1dedf63fe32fc9432d2609ec08f00916db5c2d3a775e7a7ae32992bee95 not found: ID does not exist" Jul 20 18:05:04.946896 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.946840 2567 scope.go:117] "RemoveContainer" containerID="0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86" Jul 20 18:05:04.955466 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.955450 2567 scope.go:117] "RemoveContainer" containerID="0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86" Jul 20 18:05:04.955739 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.955717 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86\": container with ID starting with 0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86 not found: ID does not exist" containerID="0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86" Jul 20 18:05:04.955780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.955746 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86"} err="failed to get container status \"0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86\": rpc error: code = NotFound desc = could not find container \"0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86\": container with ID starting with 0ea612684fe608302506ae120f77d377f1352a94f6e3e8b56df14b352864ba86 not found: ID does not exist" Jul 20 18:05:04.967074 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.967031 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" podStartSLOduration=1.967017655 podStartE2EDuration="1.967017655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:05:04.964768626 +0000 UTC m=+534.092885793" watchObservedRunningTime="2026-07-20 18:05:04.967017655 +0000 UTC m=+534.095134821" Jul 20 18:05:04.983599 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:04.983553 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" podStartSLOduration=1.983537291 podStartE2EDuration="1.983537291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:05:04.982257171 +0000 UTC m=+534.110374360" watchObservedRunningTime="2026-07-20 18:05:04.983537291 +0000 UTC m=+534.111654867" Jul 20 18:05:04.984633 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.984611 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6blx5\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6blx5" Jul 20 18:05:04.986315 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:04.986295 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cqvxk\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cqvxk" Jul 20 18:05:05.441306 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:05.441272 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="998834b8-d933-4398-a69b-c0e6bac7348f" path="/var/lib/kubelet/pods/998834b8-d933-4398-a69b-c0e6bac7348f/volumes" Jul 20 18:05:05.441645 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:05.441632 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc685221-7f49-4759-9c6c-dc02ec4c8203" path="/var/lib/kubelet/pods/dc685221-7f49-4759-9c6c-dc02ec4c8203/volumes" Jul 20 18:05:15.949587 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:15.949555 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4j4jh" Jul 20 18:05:15.950071 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:15.949623 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:21.378708 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.378670 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp"] Jul 20 18:05:21.379127 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.378926 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" podUID="cef4bdbf-1cce-4b99-afbe-768d643bf97e" containerName="manager" containerID="cri-o://0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4" gracePeriod=10 Jul 20 18:05:21.632880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.632819 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:21.714765 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.714726 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cef4bdbf-1cce-4b99-afbe-768d643bf97e-extensions-socket-volume\") pod \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " Jul 20 18:05:21.714765 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.714772 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxgwt\" (UniqueName: \"kubernetes.io/projected/cef4bdbf-1cce-4b99-afbe-768d643bf97e-kube-api-access-qxgwt\") pod \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\" (UID: \"cef4bdbf-1cce-4b99-afbe-768d643bf97e\") " Jul 20 18:05:21.715230 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.715172 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef4bdbf-1cce-4b99-afbe-768d643bf97e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cef4bdbf-1cce-4b99-afbe-768d643bf97e" (UID: "cef4bdbf-1cce-4b99-afbe-768d643bf97e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:05:21.716916 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.716894 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef4bdbf-1cce-4b99-afbe-768d643bf97e-kube-api-access-qxgwt" (OuterVolumeSpecName: "kube-api-access-qxgwt") pod "cef4bdbf-1cce-4b99-afbe-768d643bf97e" (UID: "cef4bdbf-1cce-4b99-afbe-768d643bf97e"). InnerVolumeSpecName "kube-api-access-qxgwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:05:21.816365 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.816326 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cef4bdbf-1cce-4b99-afbe-768d643bf97e-extensions-socket-volume\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:21.816365 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:21.816360 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxgwt\" (UniqueName: \"kubernetes.io/projected/cef4bdbf-1cce-4b99-afbe-768d643bf97e-kube-api-access-qxgwt\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:22.046910 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.046880 2567 generic.go:356] "Generic (PLEG): container finished" podID="cef4bdbf-1cce-4b99-afbe-768d643bf97e" containerID="0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4" exitCode=0 Jul 20 18:05:22.047066 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.046928 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" Jul 20 18:05:22.047066 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.046939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" event={"ID":"cef4bdbf-1cce-4b99-afbe-768d643bf97e","Type":"ContainerDied","Data":"0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4"} Jul 20 18:05:22.047066 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.046993 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp" event={"ID":"cef4bdbf-1cce-4b99-afbe-768d643bf97e","Type":"ContainerDied","Data":"04e78d6650486bdf2d046706a24bc94a895d31f18b927e808a847020cac041c7"} Jul 20 18:05:22.047066 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.047021 2567 scope.go:117] "RemoveContainer" containerID="0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4" Jul 20 18:05:22.056502 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.056480 2567 scope.go:117] "RemoveContainer" containerID="0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4" Jul 20 18:05:22.056763 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:22.056739 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4\": container with ID starting with 0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4 not found: ID does not exist" containerID="0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4" Jul 20 18:05:22.056840 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.056770 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4"} err="failed to get container status \"0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4\": rpc error: code = NotFound desc = could not find container \"0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4\": container with ID starting with 0892cc8e3959fb2b23ccf6fe9b03aef7910182ec4143d48f2f598cc85dbed7d4 not found: ID does not exist" Jul 20 18:05:22.070595 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.070568 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp"] Jul 20 18:05:22.077276 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:22.077253 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7bljp"] Jul 20 18:05:23.440876 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:23.440842 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef4bdbf-1cce-4b99-afbe-768d643bf97e" path="/var/lib/kubelet/pods/cef4bdbf-1cce-4b99-afbe-768d643bf97e/volumes" Jul 20 18:05:37.976683 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.976647 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd"] Jul 20 18:05:37.977165 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.977145 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef4bdbf-1cce-4b99-afbe-768d643bf97e" containerName="manager" Jul 20 18:05:37.977300 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.977170 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef4bdbf-1cce-4b99-afbe-768d643bf97e" containerName="manager" Jul 20 18:05:37.977300 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.977288 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef4bdbf-1cce-4b99-afbe-768d643bf97e" containerName="manager" Jul 20 18:05:37.979694 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.979678 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:37.982086 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.982067 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vvdmc\"" Jul 20 18:05:37.992056 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:37.992030 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd"] Jul 20 18:05:38.063007 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.062966 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063234 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063033 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063234 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063095 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063234 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063127 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4ksm\" (UniqueName: \"kubernetes.io/projected/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-kube-api-access-w4ksm\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063234 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063147 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063234 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063429 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063255 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063429 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063288 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.063429 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.063349 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164535 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164489 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164535 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164577 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164616 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164640 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164656 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4ksm\" (UniqueName: \"kubernetes.io/projected/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-kube-api-access-w4ksm\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.164794 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164743 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.165155 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.164957 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.165155 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.165002 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.165155 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.165133 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.165310 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.165172 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.165565 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.165540 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.167065 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.167040 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.167359 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.167343 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.172581 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.172562 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.172850 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.172828 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4ksm\" (UniqueName: \"kubernetes.io/projected/8c5f0746-dcec-430b-b8ea-9ced3d2a43ca-kube-api-access-w4ksm\") pod \"maas-default-gateway-openshift-default-56654dcf7-m9ntd\" (UID: \"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.290930 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.290842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:38.429209 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.429168 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd"] Jul 20 18:05:38.431319 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:05:38.431294 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5f0746_dcec_430b_b8ea_9ced3d2a43ca.slice/crio-08da1519080b5375f046fc727c35220021db4488a98f65ea3ef45034c34fa744 WatchSource:0}: Error finding container 08da1519080b5375f046fc727c35220021db4488a98f65ea3ef45034c34fa744: Status 404 returned error can't find the container with id 08da1519080b5375f046fc727c35220021db4488a98f65ea3ef45034c34fa744 Jul 20 18:05:38.433429 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.433393 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:05:38.433533 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.433474 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:05:38.433533 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:38.433517 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:05:39.147811 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:39.147773 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" event={"ID":"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca","Type":"ContainerStarted","Data":"3f0117c52a22033fa811283626b1041c157e7bb4ed77e42a092aac14c3943bfa"} Jul 20 18:05:39.148262 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:39.147825 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" event={"ID":"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca","Type":"ContainerStarted","Data":"08da1519080b5375f046fc727c35220021db4488a98f65ea3ef45034c34fa744"} Jul 20 18:05:39.169962 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:39.169911 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" podStartSLOduration=2.169895186 podStartE2EDuration="2.169895186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:05:39.168945926 +0000 UTC m=+568.297063092" watchObservedRunningTime="2026-07-20 18:05:39.169895186 +0000 UTC m=+568.298012352" Jul 20 18:05:39.291416 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:39.291385 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:39.296335 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:39.296307 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:40.153970 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.153937 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:40.155050 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.155029 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:05:40.280755 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.280718 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:05:40.283564 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.283541 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.286413 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.286392 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5hqz2\"" Jul 20 18:05:40.286544 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.286457 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 18:05:40.295561 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.295535 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:05:40.372271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.372235 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:05:40.387776 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.387741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bskzw\" (UniqueName: \"kubernetes.io/projected/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-kube-api-access-bskzw\") pod \"limitador-limitador-64c94fd64f-vj972\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.387946 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.387789 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-config-file\") pod \"limitador-limitador-64c94fd64f-vj972\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.489200 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.489093 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bskzw\" (UniqueName: \"kubernetes.io/projected/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-kube-api-access-bskzw\") pod \"limitador-limitador-64c94fd64f-vj972\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.489200 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.489150 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-config-file\") pod \"limitador-limitador-64c94fd64f-vj972\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.489783 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.489762 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-config-file\") pod \"limitador-limitador-64c94fd64f-vj972\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.498010 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.497978 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bskzw\" (UniqueName: \"kubernetes.io/projected/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-kube-api-access-bskzw\") pod \"limitador-limitador-64c94fd64f-vj972\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.594762 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.594726 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:40.741041 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:40.741014 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:05:40.743161 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:05:40.743135 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c9acfe4_f677_436b_afb3_ed5a451c5ac9.slice/crio-d79b86094e16944de50f46006ea7e911db68b9b8c158a5acbaa5de229c9af190 WatchSource:0}: Error finding container d79b86094e16944de50f46006ea7e911db68b9b8c158a5acbaa5de229c9af190: Status 404 returned error can't find the container with id d79b86094e16944de50f46006ea7e911db68b9b8c158a5acbaa5de229c9af190 Jul 20 18:05:41.074301 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.074271 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7qklt"] Jul 20 18:05:41.077443 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.077426 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:41.079888 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.079867 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gbrqk\"" Jul 20 18:05:41.085840 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.085670 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qklt"] Jul 20 18:05:41.161354 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.161313 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" event={"ID":"9c9acfe4-f677-436b-afb3-ed5a451c5ac9","Type":"ContainerStarted","Data":"d79b86094e16944de50f46006ea7e911db68b9b8c158a5acbaa5de229c9af190"} Jul 20 18:05:41.196382 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.196342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkhv2\" (UniqueName: \"kubernetes.io/projected/ba716073-f1f8-4513-8ddd-826954bcc94d-kube-api-access-nkhv2\") pod \"authorino-58cd955675-7qklt\" (UID: \"ba716073-f1f8-4513-8ddd-826954bcc94d\") " pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:41.249022 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.248986 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-d9kk4"] Jul 20 18:05:41.251954 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.251933 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:05:41.257559 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.257531 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d9kk4"] Jul 20 18:05:41.297028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.296989 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkhv2\" (UniqueName: \"kubernetes.io/projected/ba716073-f1f8-4513-8ddd-826954bcc94d-kube-api-access-nkhv2\") pod \"authorino-58cd955675-7qklt\" (UID: \"ba716073-f1f8-4513-8ddd-826954bcc94d\") " pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:41.306439 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.306406 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkhv2\" (UniqueName: \"kubernetes.io/projected/ba716073-f1f8-4513-8ddd-826954bcc94d-kube-api-access-nkhv2\") pod \"authorino-58cd955675-7qklt\" (UID: \"ba716073-f1f8-4513-8ddd-826954bcc94d\") " pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:41.388156 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.388073 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:41.398659 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.398624 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zrck\" (UniqueName: \"kubernetes.io/projected/b9895265-bf14-4a23-9279-1773872e20cf-kube-api-access-9zrck\") pod \"authorino-75d556bd77-d9kk4\" (UID: \"b9895265-bf14-4a23-9279-1773872e20cf\") " pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:05:41.499746 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.499704 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zrck\" (UniqueName: \"kubernetes.io/projected/b9895265-bf14-4a23-9279-1773872e20cf-kube-api-access-9zrck\") pod \"authorino-75d556bd77-d9kk4\" (UID: \"b9895265-bf14-4a23-9279-1773872e20cf\") " pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:05:41.509347 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.509312 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zrck\" (UniqueName: \"kubernetes.io/projected/b9895265-bf14-4a23-9279-1773872e20cf-kube-api-access-9zrck\") pod \"authorino-75d556bd77-d9kk4\" (UID: \"b9895265-bf14-4a23-9279-1773872e20cf\") " pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:05:41.526756 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.526725 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qklt"] Jul 20 18:05:41.527837 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:05:41.527816 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba716073_f1f8_4513_8ddd_826954bcc94d.slice/crio-88cafd6899a8e882387e1da8593853cf19221f01dc8064a785aac3d312baa602 WatchSource:0}: Error finding container 88cafd6899a8e882387e1da8593853cf19221f01dc8064a785aac3d312baa602: Status 404 returned error can't find the container with id 88cafd6899a8e882387e1da8593853cf19221f01dc8064a785aac3d312baa602 Jul 20 18:05:41.562369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.562338 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:05:41.697640 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:41.697609 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d9kk4"] Jul 20 18:05:41.699110 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:05:41.699086 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9895265_bf14_4a23_9279_1773872e20cf.slice/crio-adbf3331ef04a81be7368692c702ef509f3d0759aa66283d15d95ac38f6a321b WatchSource:0}: Error finding container adbf3331ef04a81be7368692c702ef509f3d0759aa66283d15d95ac38f6a321b: Status 404 returned error can't find the container with id adbf3331ef04a81be7368692c702ef509f3d0759aa66283d15d95ac38f6a321b Jul 20 18:05:42.173703 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:42.173661 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qklt" event={"ID":"ba716073-f1f8-4513-8ddd-826954bcc94d","Type":"ContainerStarted","Data":"88cafd6899a8e882387e1da8593853cf19221f01dc8064a785aac3d312baa602"} Jul 20 18:05:42.177054 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:42.176893 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d9kk4" event={"ID":"b9895265-bf14-4a23-9279-1773872e20cf","Type":"ContainerStarted","Data":"adbf3331ef04a81be7368692c702ef509f3d0759aa66283d15d95ac38f6a321b"} Jul 20 18:05:47.218194 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.218129 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d9kk4" event={"ID":"b9895265-bf14-4a23-9279-1773872e20cf","Type":"ContainerStarted","Data":"3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076"} Jul 20 18:05:47.219830 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.219807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qklt" event={"ID":"ba716073-f1f8-4513-8ddd-826954bcc94d","Type":"ContainerStarted","Data":"60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858"} Jul 20 18:05:47.221201 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.221159 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" event={"ID":"9c9acfe4-f677-436b-afb3-ed5a451c5ac9","Type":"ContainerStarted","Data":"811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e"} Jul 20 18:05:47.221331 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.221315 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:47.236350 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.236303 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-d9kk4" podStartSLOduration=1.250201104 podStartE2EDuration="6.236288733s" totalImagesPullingTime="4.986087629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:05:47.234583907 +0000 UTC m=+576.362701073" watchObservedRunningTime="2026-07-20 18:05:47.236288733 +0000 UTC m=+576.364405945" Jul 20 18:05:47.256608 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.256548 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7qklt" podStartSLOduration=1.130136079 podStartE2EDuration="6.256528109s" totalImagesPullingTime="5.12639203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:05:47.253324568 +0000 UTC m=+576.381441735" watchObservedRunningTime="2026-07-20 18:05:47.256528109 +0000 UTC m=+576.384645274" Jul 20 18:05:47.278073 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.278015 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" podStartSLOduration=1.330104556 podStartE2EDuration="7.277998484s" totalImagesPullingTime="5.947893928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:05:47.277213994 +0000 UTC m=+576.405331189" watchObservedRunningTime="2026-07-20 18:05:47.277998484 +0000 UTC m=+576.406115650" Jul 20 18:05:47.302414 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:47.302383 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qklt"] Jul 20 18:05:49.233442 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:49.233403 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7qklt" podUID="ba716073-f1f8-4513-8ddd-826954bcc94d" containerName="authorino" containerID="cri-o://60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858" gracePeriod=30 Jul 20 18:05:49.493016 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:49.492947 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:49.588988 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:49.588953 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkhv2\" (UniqueName: \"kubernetes.io/projected/ba716073-f1f8-4513-8ddd-826954bcc94d-kube-api-access-nkhv2\") pod \"ba716073-f1f8-4513-8ddd-826954bcc94d\" (UID: \"ba716073-f1f8-4513-8ddd-826954bcc94d\") " Jul 20 18:05:49.591063 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:49.591028 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba716073-f1f8-4513-8ddd-826954bcc94d-kube-api-access-nkhv2" (OuterVolumeSpecName: "kube-api-access-nkhv2") pod "ba716073-f1f8-4513-8ddd-826954bcc94d" (UID: "ba716073-f1f8-4513-8ddd-826954bcc94d"). InnerVolumeSpecName "kube-api-access-nkhv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:05:49.689886 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:49.689849 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkhv2\" (UniqueName: \"kubernetes.io/projected/ba716073-f1f8-4513-8ddd-826954bcc94d-kube-api-access-nkhv2\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:50.240713 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.240614 2567 generic.go:356] "Generic (PLEG): container finished" podID="ba716073-f1f8-4513-8ddd-826954bcc94d" containerID="60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858" exitCode=0 Jul 20 18:05:50.240713 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.240671 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qklt" Jul 20 18:05:50.240713 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.240703 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qklt" event={"ID":"ba716073-f1f8-4513-8ddd-826954bcc94d","Type":"ContainerDied","Data":"60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858"} Jul 20 18:05:50.241174 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.240748 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qklt" event={"ID":"ba716073-f1f8-4513-8ddd-826954bcc94d","Type":"ContainerDied","Data":"88cafd6899a8e882387e1da8593853cf19221f01dc8064a785aac3d312baa602"} Jul 20 18:05:50.241174 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.240768 2567 scope.go:117] "RemoveContainer" containerID="60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858" Jul 20 18:05:50.249905 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.249890 2567 scope.go:117] "RemoveContainer" containerID="60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858" Jul 20 18:05:50.250221 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:50.250200 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858\": container with ID starting with 60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858 not found: ID does not exist" containerID="60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858" Jul 20 18:05:50.250286 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.250231 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858"} err="failed to get container status \"60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858\": rpc error: code = NotFound desc = could not find container \"60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858\": container with ID starting with 60c220a2cc6401d1a15b2ab00daf85dbab1bdbdf09f494a5d1144f170d530858 not found: ID does not exist" Jul 20 18:05:50.262415 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.262389 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qklt"] Jul 20 18:05:50.268505 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:50.268481 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qklt"] Jul 20 18:05:51.440585 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:51.440549 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba716073-f1f8-4513-8ddd-826954bcc94d" path="/var/lib/kubelet/pods/ba716073-f1f8-4513-8ddd-826954bcc94d/volumes" Jul 20 18:05:58.229344 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:58.229312 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:58.295725 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:58.295692 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:05:58.296161 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:58.296122 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" podUID="9c9acfe4-f677-436b-afb3-ed5a451c5ac9" containerName="limitador" containerID="cri-o://811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e" gracePeriod=30 Jul 20 18:05:59.258534 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.258510 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:59.294969 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.294935 2567 generic.go:356] "Generic (PLEG): container finished" podID="9c9acfe4-f677-436b-afb3-ed5a451c5ac9" containerID="811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e" exitCode=0 Jul 20 18:05:59.295167 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.294987 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" Jul 20 18:05:59.295167 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.295015 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" event={"ID":"9c9acfe4-f677-436b-afb3-ed5a451c5ac9","Type":"ContainerDied","Data":"811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e"} Jul 20 18:05:59.295167 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.295065 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vj972" event={"ID":"9c9acfe4-f677-436b-afb3-ed5a451c5ac9","Type":"ContainerDied","Data":"d79b86094e16944de50f46006ea7e911db68b9b8c158a5acbaa5de229c9af190"} Jul 20 18:05:59.295167 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.295087 2567 scope.go:117] "RemoveContainer" containerID="811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e" Jul 20 18:05:59.304687 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.304512 2567 scope.go:117] "RemoveContainer" containerID="811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e" Jul 20 18:05:59.304885 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:05:59.304866 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e\": container with ID starting with 811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e not found: ID does not exist" containerID="811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e" Jul 20 18:05:59.304949 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.304900 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e"} err="failed to get container status \"811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e\": rpc error: code = NotFound desc = could not find container \"811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e\": container with ID starting with 811a59c39394f12831cb268b8dcea8957cc313a0d7afbfa2f16d052a6990d38e not found: ID does not exist" Jul 20 18:05:59.374218 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.374114 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-config-file\") pod \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " Jul 20 18:05:59.374372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.374237 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bskzw\" (UniqueName: \"kubernetes.io/projected/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-kube-api-access-bskzw\") pod \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\" (UID: \"9c9acfe4-f677-436b-afb3-ed5a451c5ac9\") " Jul 20 18:05:59.374595 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.374567 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-config-file" (OuterVolumeSpecName: "config-file") pod "9c9acfe4-f677-436b-afb3-ed5a451c5ac9" (UID: "9c9acfe4-f677-436b-afb3-ed5a451c5ac9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:05:59.376281 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.376261 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-kube-api-access-bskzw" (OuterVolumeSpecName: "kube-api-access-bskzw") pod "9c9acfe4-f677-436b-afb3-ed5a451c5ac9" (UID: "9c9acfe4-f677-436b-afb3-ed5a451c5ac9"). InnerVolumeSpecName "kube-api-access-bskzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:05:59.475364 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.475329 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bskzw\" (UniqueName: \"kubernetes.io/projected/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-kube-api-access-bskzw\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:59.475364 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.475358 2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c9acfe4-f677-436b-afb3-ed5a451c5ac9-config-file\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:05:59.614590 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.614558 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:05:59.618315 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:05:59.618288 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vj972"] Jul 20 18:06:01.441059 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:01.441028 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c9acfe4-f677-436b-afb3-ed5a451c5ac9" path="/var/lib/kubelet/pods/9c9acfe4-f677-436b-afb3-ed5a451c5ac9/volumes" Jul 20 18:06:02.239083 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239047 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wb78n"] Jul 20 18:06:02.239464 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239451 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c9acfe4-f677-436b-afb3-ed5a451c5ac9" containerName="limitador" Jul 20 18:06:02.239512 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239466 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c9acfe4-f677-436b-afb3-ed5a451c5ac9" containerName="limitador" Jul 20 18:06:02.239512 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239494 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba716073-f1f8-4513-8ddd-826954bcc94d" containerName="authorino" Jul 20 18:06:02.239512 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239500 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba716073-f1f8-4513-8ddd-826954bcc94d" containerName="authorino" Jul 20 18:06:02.239604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239556 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c9acfe4-f677-436b-afb3-ed5a451c5ac9" containerName="limitador" Jul 20 18:06:02.239604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.239565 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba716073-f1f8-4513-8ddd-826954bcc94d" containerName="authorino" Jul 20 18:06:02.242694 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.242667 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.245797 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.245776 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 20 18:06:02.245938 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.245810 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 18:06:02.246379 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.246357 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-664x9\"" Jul 20 18:06:02.246379 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.246368 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 20 18:06:02.252736 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.252714 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wb78n"] Jul 20 18:06:02.303422 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.303382 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fndh\" (UniqueName: \"kubernetes.io/projected/1533c7d0-ebd6-453a-8465-4263a389e422-kube-api-access-5fndh\") pod \"postgres-69fb4594cf-wb78n\" (UID: \"1533c7d0-ebd6-453a-8465-4263a389e422\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.303603 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.303480 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1533c7d0-ebd6-453a-8465-4263a389e422-data\") pod \"postgres-69fb4594cf-wb78n\" (UID: \"1533c7d0-ebd6-453a-8465-4263a389e422\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.404857 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.404815 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1533c7d0-ebd6-453a-8465-4263a389e422-data\") pod \"postgres-69fb4594cf-wb78n\" (UID: \"1533c7d0-ebd6-453a-8465-4263a389e422\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.405042 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.404892 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fndh\" (UniqueName: \"kubernetes.io/projected/1533c7d0-ebd6-453a-8465-4263a389e422-kube-api-access-5fndh\") pod \"postgres-69fb4594cf-wb78n\" (UID: \"1533c7d0-ebd6-453a-8465-4263a389e422\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.405265 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.405247 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1533c7d0-ebd6-453a-8465-4263a389e422-data\") pod \"postgres-69fb4594cf-wb78n\" (UID: \"1533c7d0-ebd6-453a-8465-4263a389e422\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.415258 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.415228 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fndh\" (UniqueName: \"kubernetes.io/projected/1533c7d0-ebd6-453a-8465-4263a389e422-kube-api-access-5fndh\") pod \"postgres-69fb4594cf-wb78n\" (UID: \"1533c7d0-ebd6-453a-8465-4263a389e422\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.554770 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.554733 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:02.695710 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:02.695680 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wb78n"] Jul 20 18:06:02.696820 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:06:02.696792 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1533c7d0_ebd6_453a_8465_4263a389e422.slice/crio-b31fb1635f9c9d293ed90dcb34301d2b3b262ee9be081f9ab82780dfb20a2192 WatchSource:0}: Error finding container b31fb1635f9c9d293ed90dcb34301d2b3b262ee9be081f9ab82780dfb20a2192: Status 404 returned error can't find the container with id b31fb1635f9c9d293ed90dcb34301d2b3b262ee9be081f9ab82780dfb20a2192 Jul 20 18:06:03.320875 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:03.320840 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" event={"ID":"1533c7d0-ebd6-453a-8465-4263a389e422","Type":"ContainerStarted","Data":"b31fb1635f9c9d293ed90dcb34301d2b3b262ee9be081f9ab82780dfb20a2192"} Jul 20 18:06:08.356732 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:08.356691 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" event={"ID":"1533c7d0-ebd6-453a-8465-4263a389e422","Type":"ContainerStarted","Data":"15ac04f9f5794e98bb0035717c12bbecb33f880222a7ecf9a258c80b372993d0"} Jul 20 18:06:08.357121 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:08.356760 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:08.375695 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:08.375641 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" podStartSLOduration=1.148361604 podStartE2EDuration="6.375625292s" totalImagesPullingTime="5.227263688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:06:08.372816904 +0000 UTC m=+597.500934085" watchObservedRunningTime="2026-07-20 18:06:08.375625292 +0000 UTC m=+597.503742761" Jul 20 18:06:11.313665 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:11.313638 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:06:11.314305 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:11.314285 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:06:14.395332 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:14.395299 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wb78n" Jul 20 18:06:14.972912 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:14.972864 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7w9tp"] Jul 20 18:06:14.978054 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:14.978027 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:14.984267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:14.984235 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7w9tp"] Jul 20 18:06:15.124890 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.124851 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st9jr\" (UniqueName: \"kubernetes.io/projected/cdef4f8e-6fe3-4c4c-be83-b60cea30987f-kube-api-access-st9jr\") pod \"authorino-5d4b67c446-7w9tp\" (UID: \"cdef4f8e-6fe3-4c4c-be83-b60cea30987f\") " pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:15.225642 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.225543 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st9jr\" (UniqueName: \"kubernetes.io/projected/cdef4f8e-6fe3-4c4c-be83-b60cea30987f-kube-api-access-st9jr\") pod \"authorino-5d4b67c446-7w9tp\" (UID: \"cdef4f8e-6fe3-4c4c-be83-b60cea30987f\") " pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:15.225964 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.225944 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7w9tp"] Jul 20 18:06:15.226252 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:06:15.226231 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-st9jr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-7w9tp" podUID="cdef4f8e-6fe3-4c4c-be83-b60cea30987f" Jul 20 18:06:15.235652 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.235621 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st9jr\" (UniqueName: \"kubernetes.io/projected/cdef4f8e-6fe3-4c4c-be83-b60cea30987f-kube-api-access-st9jr\") pod \"authorino-5d4b67c446-7w9tp\" (UID: \"cdef4f8e-6fe3-4c4c-be83-b60cea30987f\") " pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:15.256431 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.256404 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84d5478c67-9k2tb"] Jul 20 18:06:15.259375 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.259352 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:15.269142 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.269119 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84d5478c67-9k2tb"] Jul 20 18:06:15.327152 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.327122 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxgxh\" (UniqueName: \"kubernetes.io/projected/9a8b0159-3d1e-4374-8218-155d8a17a062-kube-api-access-cxgxh\") pod \"authorino-84d5478c67-9k2tb\" (UID: \"9a8b0159-3d1e-4374-8218-155d8a17a062\") " pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:15.338302 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.338273 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84d5478c67-9k2tb"] Jul 20 18:06:15.338633 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:06:15.338612 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cxgxh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-84d5478c67-9k2tb" podUID="9a8b0159-3d1e-4374-8218-155d8a17a062" Jul 20 18:06:15.370013 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.369983 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d56b7dfdc-djsn4"] Jul 20 18:06:15.372665 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.372647 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.375223 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.375195 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 20 18:06:15.380728 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.380706 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d56b7dfdc-djsn4"] Jul 20 18:06:15.409654 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.409622 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:15.410028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.409623 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:15.415827 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.415804 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:15.418984 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.418969 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:15.428298 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.428274 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxgxh\" (UniqueName: \"kubernetes.io/projected/9a8b0159-3d1e-4374-8218-155d8a17a062-kube-api-access-cxgxh\") pod \"authorino-84d5478c67-9k2tb\" (UID: \"9a8b0159-3d1e-4374-8218-155d8a17a062\") " pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:15.436320 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.436295 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxgxh\" (UniqueName: \"kubernetes.io/projected/9a8b0159-3d1e-4374-8218-155d8a17a062-kube-api-access-cxgxh\") pod \"authorino-84d5478c67-9k2tb\" (UID: \"9a8b0159-3d1e-4374-8218-155d8a17a062\") " pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:15.529583 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.529491 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st9jr\" (UniqueName: \"kubernetes.io/projected/cdef4f8e-6fe3-4c4c-be83-b60cea30987f-kube-api-access-st9jr\") pod \"cdef4f8e-6fe3-4c4c-be83-b60cea30987f\" (UID: \"cdef4f8e-6fe3-4c4c-be83-b60cea30987f\") " Jul 20 18:06:15.529583 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.529570 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxgxh\" (UniqueName: \"kubernetes.io/projected/9a8b0159-3d1e-4374-8218-155d8a17a062-kube-api-access-cxgxh\") pod \"9a8b0159-3d1e-4374-8218-155d8a17a062\" (UID: \"9a8b0159-3d1e-4374-8218-155d8a17a062\") " Jul 20 18:06:15.529825 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.529722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq9zq\" (UniqueName: \"kubernetes.io/projected/bfa41074-8909-43fa-bf62-a1863ca43d6d-kube-api-access-xq9zq\") pod \"authorino-7d56b7dfdc-djsn4\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.529825 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.529772 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa41074-8909-43fa-bf62-a1863ca43d6d-tls-cert\") pod \"authorino-7d56b7dfdc-djsn4\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.531754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.531717 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdef4f8e-6fe3-4c4c-be83-b60cea30987f-kube-api-access-st9jr" (OuterVolumeSpecName: "kube-api-access-st9jr") pod "cdef4f8e-6fe3-4c4c-be83-b60cea30987f" (UID: "cdef4f8e-6fe3-4c4c-be83-b60cea30987f"). InnerVolumeSpecName "kube-api-access-st9jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:06:15.531754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.531739 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a8b0159-3d1e-4374-8218-155d8a17a062-kube-api-access-cxgxh" (OuterVolumeSpecName: "kube-api-access-cxgxh") pod "9a8b0159-3d1e-4374-8218-155d8a17a062" (UID: "9a8b0159-3d1e-4374-8218-155d8a17a062"). InnerVolumeSpecName "kube-api-access-cxgxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:06:15.630440 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.630400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq9zq\" (UniqueName: \"kubernetes.io/projected/bfa41074-8909-43fa-bf62-a1863ca43d6d-kube-api-access-xq9zq\") pod \"authorino-7d56b7dfdc-djsn4\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.630632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.630457 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa41074-8909-43fa-bf62-a1863ca43d6d-tls-cert\") pod \"authorino-7d56b7dfdc-djsn4\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.630632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.630573 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st9jr\" (UniqueName: \"kubernetes.io/projected/cdef4f8e-6fe3-4c4c-be83-b60cea30987f-kube-api-access-st9jr\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:06:15.630632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.630590 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxgxh\" (UniqueName: \"kubernetes.io/projected/9a8b0159-3d1e-4374-8218-155d8a17a062-kube-api-access-cxgxh\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:06:15.632989 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.632961 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa41074-8909-43fa-bf62-a1863ca43d6d-tls-cert\") pod \"authorino-7d56b7dfdc-djsn4\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.639434 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.639404 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq9zq\" (UniqueName: \"kubernetes.io/projected/bfa41074-8909-43fa-bf62-a1863ca43d6d-kube-api-access-xq9zq\") pod \"authorino-7d56b7dfdc-djsn4\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.682623 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.682592 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:06:15.819932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:15.819899 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d56b7dfdc-djsn4"] Jul 20 18:06:15.821901 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:06:15.821874 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfa41074_8909_43fa_bf62_a1863ca43d6d.slice/crio-6bcab814b13b2914882083a0b2391fdb83c322b0a2fb6398c784787388799d00 WatchSource:0}: Error finding container 6bcab814b13b2914882083a0b2391fdb83c322b0a2fb6398c784787388799d00: Status 404 returned error can't find the container with id 6bcab814b13b2914882083a0b2391fdb83c322b0a2fb6398c784787388799d00 Jul 20 18:06:16.416402 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.416365 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" event={"ID":"bfa41074-8909-43fa-bf62-a1863ca43d6d","Type":"ContainerStarted","Data":"6bcab814b13b2914882083a0b2391fdb83c322b0a2fb6398c784787388799d00"} Jul 20 18:06:16.416402 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.416392 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d5478c67-9k2tb" Jul 20 18:06:16.416936 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.416392 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7w9tp" Jul 20 18:06:16.456026 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.455988 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7w9tp"] Jul 20 18:06:16.464271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.464242 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7w9tp"] Jul 20 18:06:16.491271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.491245 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84d5478c67-9k2tb"] Jul 20 18:06:16.496823 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:16.496795 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84d5478c67-9k2tb"] Jul 20 18:06:17.423341 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.423299 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" event={"ID":"bfa41074-8909-43fa-bf62-a1863ca43d6d","Type":"ContainerStarted","Data":"29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5"} Jul 20 18:06:17.441493 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.441459 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a8b0159-3d1e-4374-8218-155d8a17a062" path="/var/lib/kubelet/pods/9a8b0159-3d1e-4374-8218-155d8a17a062/volumes" Jul 20 18:06:17.441702 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.441662 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" podStartSLOduration=1.832729737 podStartE2EDuration="2.441643935s" totalImagesPullingTime="608.914198ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:06:17.440434829 +0000 UTC m=+606.568551997" watchObservedRunningTime="2026-07-20 18:06:17.441643935 +0000 UTC m=+606.569761102" Jul 20 18:06:17.441808 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.441790 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdef4f8e-6fe3-4c4c-be83-b60cea30987f" path="/var/lib/kubelet/pods/cdef4f8e-6fe3-4c4c-be83-b60cea30987f/volumes" Jul 20 18:06:17.473208 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.473155 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d9kk4"] Jul 20 18:06:17.473395 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.473351 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-d9kk4" podUID="b9895265-bf14-4a23-9279-1773872e20cf" containerName="authorino" containerID="cri-o://3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076" gracePeriod=30 Jul 20 18:06:17.755981 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.755890 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:06:17.851874 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.851835 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zrck\" (UniqueName: \"kubernetes.io/projected/b9895265-bf14-4a23-9279-1773872e20cf-kube-api-access-9zrck\") pod \"b9895265-bf14-4a23-9279-1773872e20cf\" (UID: \"b9895265-bf14-4a23-9279-1773872e20cf\") " Jul 20 18:06:17.853910 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.853873 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9895265-bf14-4a23-9279-1773872e20cf-kube-api-access-9zrck" (OuterVolumeSpecName: "kube-api-access-9zrck") pod "b9895265-bf14-4a23-9279-1773872e20cf" (UID: "b9895265-bf14-4a23-9279-1773872e20cf"). InnerVolumeSpecName "kube-api-access-9zrck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:06:17.953029 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:17.952991 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zrck\" (UniqueName: \"kubernetes.io/projected/b9895265-bf14-4a23-9279-1773872e20cf-kube-api-access-9zrck\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:06:18.431294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.431260 2567 generic.go:356] "Generic (PLEG): container finished" podID="b9895265-bf14-4a23-9279-1773872e20cf" containerID="3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076" exitCode=0 Jul 20 18:06:18.431867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.431304 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d9kk4" Jul 20 18:06:18.431867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.431348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d9kk4" event={"ID":"b9895265-bf14-4a23-9279-1773872e20cf","Type":"ContainerDied","Data":"3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076"} Jul 20 18:06:18.431867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.431386 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d9kk4" event={"ID":"b9895265-bf14-4a23-9279-1773872e20cf","Type":"ContainerDied","Data":"adbf3331ef04a81be7368692c702ef509f3d0759aa66283d15d95ac38f6a321b"} Jul 20 18:06:18.431867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.431407 2567 scope.go:117] "RemoveContainer" containerID="3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076" Jul 20 18:06:18.441537 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.441518 2567 scope.go:117] "RemoveContainer" containerID="3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076" Jul 20 18:06:18.441808 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:06:18.441787 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076\": container with ID starting with 3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076 not found: ID does not exist" containerID="3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076" Jul 20 18:06:18.441886 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.441816 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076"} err="failed to get container status \"3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076\": rpc error: code = NotFound desc = could not find container \"3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076\": container with ID starting with 3aaffade9549e056a4f70d202ce0ddb73ceedabc253687fb235411fe01ad3076 not found: ID does not exist" Jul 20 18:06:18.455132 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.455092 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d9kk4"] Jul 20 18:06:18.459313 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:18.459287 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d9kk4"] Jul 20 18:06:19.440848 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:19.440814 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9895265-bf14-4a23-9279-1773872e20cf" path="/var/lib/kubelet/pods/b9895265-bf14-4a23-9279-1773872e20cf/volumes" Jul 20 18:06:22.587469 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.587434 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-868dcb8c45-22pxl"] Jul 20 18:06:22.587899 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.587847 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9895265-bf14-4a23-9279-1773872e20cf" containerName="authorino" Jul 20 18:06:22.587899 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.587860 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9895265-bf14-4a23-9279-1773872e20cf" containerName="authorino" Jul 20 18:06:22.587978 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.587939 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9895265-bf14-4a23-9279-1773872e20cf" containerName="authorino" Jul 20 18:06:22.611357 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.611325 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-868dcb8c45-22pxl"] Jul 20 18:06:22.611520 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.611429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.614703 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.614674 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-gfll9\"" Jul 20 18:06:22.614839 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.614811 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 18:06:22.615410 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.615392 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 18:06:22.697980 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.697940 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqzl6\" (UniqueName: \"kubernetes.io/projected/8423f3b1-de1b-461c-88be-30014e3b9b5d-kube-api-access-cqzl6\") pod \"maas-controller-868dcb8c45-22pxl\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.697980 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.697978 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8423f3b1-de1b-461c-88be-30014e3b9b5d-webhook-cert\") pod \"maas-controller-868dcb8c45-22pxl\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.774482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.774435 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bd97c8594-bcxw9"] Jul 20 18:06:22.777007 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.776988 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:22.787037 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.787008 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bd97c8594-bcxw9"] Jul 20 18:06:22.798663 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.798627 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8423f3b1-de1b-461c-88be-30014e3b9b5d-webhook-cert\") pod \"maas-controller-868dcb8c45-22pxl\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.798881 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.798851 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzl6\" (UniqueName: \"kubernetes.io/projected/8423f3b1-de1b-461c-88be-30014e3b9b5d-kube-api-access-cqzl6\") pod \"maas-controller-868dcb8c45-22pxl\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.801383 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.801363 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8423f3b1-de1b-461c-88be-30014e3b9b5d-webhook-cert\") pod \"maas-controller-868dcb8c45-22pxl\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.816173 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.816145 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzl6\" (UniqueName: \"kubernetes.io/projected/8423f3b1-de1b-461c-88be-30014e3b9b5d-kube-api-access-cqzl6\") pod \"maas-controller-868dcb8c45-22pxl\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:22.900411 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.900308 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh7qb\" (UniqueName: \"kubernetes.io/projected/8f16886b-e348-4d7e-ad04-7ef070593d3a-kube-api-access-zh7qb\") pod \"maas-controller-7bd97c8594-bcxw9\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:22.900411 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.900378 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f16886b-e348-4d7e-ad04-7ef070593d3a-webhook-cert\") pod \"maas-controller-7bd97c8594-bcxw9\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:22.922236 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:22.922171 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:23.001813 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.001774 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh7qb\" (UniqueName: \"kubernetes.io/projected/8f16886b-e348-4d7e-ad04-7ef070593d3a-kube-api-access-zh7qb\") pod \"maas-controller-7bd97c8594-bcxw9\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:23.001991 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.001864 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f16886b-e348-4d7e-ad04-7ef070593d3a-webhook-cert\") pod \"maas-controller-7bd97c8594-bcxw9\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:23.004254 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.004229 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f16886b-e348-4d7e-ad04-7ef070593d3a-webhook-cert\") pod \"maas-controller-7bd97c8594-bcxw9\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:23.012554 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.012004 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh7qb\" (UniqueName: \"kubernetes.io/projected/8f16886b-e348-4d7e-ad04-7ef070593d3a-kube-api-access-zh7qb\") pod \"maas-controller-7bd97c8594-bcxw9\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:23.062686 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.062656 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-868dcb8c45-22pxl"] Jul 20 18:06:23.064122 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:06:23.064087 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8423f3b1_de1b_461c_88be_30014e3b9b5d.slice/crio-10fb67e8ad444daaba4e1557163088b853321c9755d66622e9887a5cdff99dfa WatchSource:0}: Error finding container 10fb67e8ad444daaba4e1557163088b853321c9755d66622e9887a5cdff99dfa: Status 404 returned error can't find the container with id 10fb67e8ad444daaba4e1557163088b853321c9755d66622e9887a5cdff99dfa Jul 20 18:06:23.089595 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.089564 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:23.234323 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.234293 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bd97c8594-bcxw9"] Jul 20 18:06:23.235604 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:06:23.235573 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f16886b_e348_4d7e_ad04_7ef070593d3a.slice/crio-d0eedc265a7d398d4c0cfd5283363c2818e57289dd53979708baea1d4cea72ad WatchSource:0}: Error finding container d0eedc265a7d398d4c0cfd5283363c2818e57289dd53979708baea1d4cea72ad: Status 404 returned error can't find the container with id d0eedc265a7d398d4c0cfd5283363c2818e57289dd53979708baea1d4cea72ad Jul 20 18:06:23.461222 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.461101 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" event={"ID":"8f16886b-e348-4d7e-ad04-7ef070593d3a","Type":"ContainerStarted","Data":"d0eedc265a7d398d4c0cfd5283363c2818e57289dd53979708baea1d4cea72ad"} Jul 20 18:06:23.462240 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:23.462216 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-868dcb8c45-22pxl" event={"ID":"8423f3b1-de1b-461c-88be-30014e3b9b5d","Type":"ContainerStarted","Data":"10fb67e8ad444daaba4e1557163088b853321c9755d66622e9887a5cdff99dfa"} Jul 20 18:06:26.483348 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:26.483297 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-868dcb8c45-22pxl" event={"ID":"8423f3b1-de1b-461c-88be-30014e3b9b5d","Type":"ContainerStarted","Data":"c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a"} Jul 20 18:06:26.483831 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:26.483400 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:26.485171 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:26.485146 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" event={"ID":"8f16886b-e348-4d7e-ad04-7ef070593d3a","Type":"ContainerStarted","Data":"c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad"} Jul 20 18:06:26.485338 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:26.485295 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:26.505336 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:26.505278 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-868dcb8c45-22pxl" podStartSLOduration=1.915243189 podStartE2EDuration="4.505262854s" totalImagesPullingTime="2.590019665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:06:26.503900889 +0000 UTC m=+615.632018055" watchObservedRunningTime="2026-07-20 18:06:26.505262854 +0000 UTC m=+615.633380020" Jul 20 18:06:26.526731 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:26.526678 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" podStartSLOduration=2.118954927 podStartE2EDuration="4.526663112s" totalImagesPullingTime="2.407708185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:06:26.524910625 +0000 UTC m=+615.653027792" watchObservedRunningTime="2026-07-20 18:06:26.526663112 +0000 UTC m=+615.654780281" Jul 20 18:06:37.494728 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.494649 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:37.496138 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.494932 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:06:37.558769 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.558738 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-868dcb8c45-22pxl"] Jul 20 18:06:37.558961 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.558925 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-868dcb8c45-22pxl" podUID="8423f3b1-de1b-461c-88be-30014e3b9b5d" containerName="manager" containerID="cri-o://c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a" gracePeriod=30 Jul 20 18:06:37.809169 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.809137 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:37.843876 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.843840 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqzl6\" (UniqueName: \"kubernetes.io/projected/8423f3b1-de1b-461c-88be-30014e3b9b5d-kube-api-access-cqzl6\") pod \"8423f3b1-de1b-461c-88be-30014e3b9b5d\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " Jul 20 18:06:37.844060 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.843891 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8423f3b1-de1b-461c-88be-30014e3b9b5d-webhook-cert\") pod \"8423f3b1-de1b-461c-88be-30014e3b9b5d\" (UID: \"8423f3b1-de1b-461c-88be-30014e3b9b5d\") " Jul 20 18:06:37.846153 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.846121 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8423f3b1-de1b-461c-88be-30014e3b9b5d-kube-api-access-cqzl6" (OuterVolumeSpecName: "kube-api-access-cqzl6") pod "8423f3b1-de1b-461c-88be-30014e3b9b5d" (UID: "8423f3b1-de1b-461c-88be-30014e3b9b5d"). InnerVolumeSpecName "kube-api-access-cqzl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:06:37.846153 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.846127 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8423f3b1-de1b-461c-88be-30014e3b9b5d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8423f3b1-de1b-461c-88be-30014e3b9b5d" (UID: "8423f3b1-de1b-461c-88be-30014e3b9b5d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:06:37.945207 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.945145 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqzl6\" (UniqueName: \"kubernetes.io/projected/8423f3b1-de1b-461c-88be-30014e3b9b5d-kube-api-access-cqzl6\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:06:37.945207 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:37.945192 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8423f3b1-de1b-461c-88be-30014e3b9b5d-webhook-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:06:38.555404 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.555364 2567 generic.go:356] "Generic (PLEG): container finished" podID="8423f3b1-de1b-461c-88be-30014e3b9b5d" containerID="c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a" exitCode=0 Jul 20 18:06:38.555809 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.555410 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-868dcb8c45-22pxl" Jul 20 18:06:38.555809 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.555443 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-868dcb8c45-22pxl" event={"ID":"8423f3b1-de1b-461c-88be-30014e3b9b5d","Type":"ContainerDied","Data":"c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a"} Jul 20 18:06:38.555809 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.555499 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-868dcb8c45-22pxl" event={"ID":"8423f3b1-de1b-461c-88be-30014e3b9b5d","Type":"ContainerDied","Data":"10fb67e8ad444daaba4e1557163088b853321c9755d66622e9887a5cdff99dfa"} Jul 20 18:06:38.555809 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.555521 2567 scope.go:117] "RemoveContainer" containerID="c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a" Jul 20 18:06:38.565924 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.565733 2567 scope.go:117] "RemoveContainer" containerID="c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a" Jul 20 18:06:38.566015 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:06:38.565996 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a\": container with ID starting with c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a not found: ID does not exist" containerID="c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a" Jul 20 18:06:38.566056 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.566026 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a"} err="failed to get container status \"c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a\": rpc error: code = NotFound desc = could not find container \"c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a\": container with ID starting with c94fdee337b816647f8e0d89aa84e0a462c0634487b7ba2c8e8a01d29a99556a not found: ID does not exist" Jul 20 18:06:38.583113 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.579133 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-868dcb8c45-22pxl"] Jul 20 18:06:38.588829 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:38.588805 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-868dcb8c45-22pxl"] Jul 20 18:06:39.441034 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:39.441002 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8423f3b1-de1b-461c-88be-30014e3b9b5d" path="/var/lib/kubelet/pods/8423f3b1-de1b-461c-88be-30014e3b9b5d/volumes" Jul 20 18:06:56.573624 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.573584 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wp2ks"] Jul 20 18:06:56.574165 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.574146 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8423f3b1-de1b-461c-88be-30014e3b9b5d" containerName="manager" Jul 20 18:06:56.574255 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.574171 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8423f3b1-de1b-461c-88be-30014e3b9b5d" containerName="manager" Jul 20 18:06:56.574301 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.574277 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8423f3b1-de1b-461c-88be-30014e3b9b5d" containerName="manager" Jul 20 18:06:56.578190 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.578156 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.580867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.580845 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 20 18:06:56.593003 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.592972 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-5nm5f"] Jul 20 18:06:56.596477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.596447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.596901 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.596874 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wp2ks"] Jul 20 18:06:56.599537 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.599519 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8vhgj\"" Jul 20 18:06:56.613068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.613035 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-5nm5f"] Jul 20 18:06:56.725195 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.725146 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b9575a6-2fca-43e5-b804-1e5e32f9dc68-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wp2ks\" (UID: \"9b9575a6-2fca-43e5-b804-1e5e32f9dc68\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.725383 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.725231 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jx6x\" (UniqueName: \"kubernetes.io/projected/9b9575a6-2fca-43e5-b804-1e5e32f9dc68-kube-api-access-6jx6x\") pod \"payload-pre-processing-7677b774bb-wp2ks\" (UID: \"9b9575a6-2fca-43e5-b804-1e5e32f9dc68\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.725383 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.725257 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnq9m\" (UniqueName: \"kubernetes.io/projected/94b2cfc1-12b2-4d23-b444-63ccc12c6f06-kube-api-access-pnq9m\") pod \"payload-processing-5bf47cd988-5nm5f\" (UID: \"94b2cfc1-12b2-4d23-b444-63ccc12c6f06\") " pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.725383 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.725312 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94b2cfc1-12b2-4d23-b444-63ccc12c6f06-plugins-config-volume\") pod \"payload-processing-5bf47cd988-5nm5f\" (UID: \"94b2cfc1-12b2-4d23-b444-63ccc12c6f06\") " pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.826719 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.826638 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94b2cfc1-12b2-4d23-b444-63ccc12c6f06-plugins-config-volume\") pod \"payload-processing-5bf47cd988-5nm5f\" (UID: \"94b2cfc1-12b2-4d23-b444-63ccc12c6f06\") " pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.826719 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.826697 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b9575a6-2fca-43e5-b804-1e5e32f9dc68-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wp2ks\" (UID: \"9b9575a6-2fca-43e5-b804-1e5e32f9dc68\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.826950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.826737 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jx6x\" (UniqueName: \"kubernetes.io/projected/9b9575a6-2fca-43e5-b804-1e5e32f9dc68-kube-api-access-6jx6x\") pod \"payload-pre-processing-7677b774bb-wp2ks\" (UID: \"9b9575a6-2fca-43e5-b804-1e5e32f9dc68\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.826950 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.826755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnq9m\" (UniqueName: \"kubernetes.io/projected/94b2cfc1-12b2-4d23-b444-63ccc12c6f06-kube-api-access-pnq9m\") pod \"payload-processing-5bf47cd988-5nm5f\" (UID: \"94b2cfc1-12b2-4d23-b444-63ccc12c6f06\") " pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.827482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.827462 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94b2cfc1-12b2-4d23-b444-63ccc12c6f06-plugins-config-volume\") pod \"payload-processing-5bf47cd988-5nm5f\" (UID: \"94b2cfc1-12b2-4d23-b444-63ccc12c6f06\") " pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.827527 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.827465 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b9575a6-2fca-43e5-b804-1e5e32f9dc68-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wp2ks\" (UID: \"9b9575a6-2fca-43e5-b804-1e5e32f9dc68\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.835136 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.835109 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnq9m\" (UniqueName: \"kubernetes.io/projected/94b2cfc1-12b2-4d23-b444-63ccc12c6f06-kube-api-access-pnq9m\") pod \"payload-processing-5bf47cd988-5nm5f\" (UID: \"94b2cfc1-12b2-4d23-b444-63ccc12c6f06\") " pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:56.835398 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.835377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jx6x\" (UniqueName: \"kubernetes.io/projected/9b9575a6-2fca-43e5-b804-1e5e32f9dc68-kube-api-access-6jx6x\") pod \"payload-pre-processing-7677b774bb-wp2ks\" (UID: \"9b9575a6-2fca-43e5-b804-1e5e32f9dc68\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.889279 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.889246 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:56.908354 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:56.908328 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:57.046537 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:57.046397 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wp2ks"] Jul 20 18:06:57.047729 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:06:57.047693 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b9575a6_2fca_43e5_b804_1e5e32f9dc68.slice/crio-c5f47ef8c681e9c64dc2ffc10a391ef33f7257a020f1435d0488d7a7f2e1e7a3 WatchSource:0}: Error finding container c5f47ef8c681e9c64dc2ffc10a391ef33f7257a020f1435d0488d7a7f2e1e7a3: Status 404 returned error can't find the container with id c5f47ef8c681e9c64dc2ffc10a391ef33f7257a020f1435d0488d7a7f2e1e7a3 Jul 20 18:06:57.049933 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:57.049914 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 18:06:57.281271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:57.279039 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-5nm5f"] Jul 20 18:06:57.286463 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:06:57.286434 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b2cfc1_12b2_4d23_b444_63ccc12c6f06.slice/crio-a5d5c3411c12ecf94e469a369ef721459639d99af886a1db52d15a36f06dfe81 WatchSource:0}: Error finding container a5d5c3411c12ecf94e469a369ef721459639d99af886a1db52d15a36f06dfe81: Status 404 returned error can't find the container with id a5d5c3411c12ecf94e469a369ef721459639d99af886a1db52d15a36f06dfe81 Jul 20 18:06:57.671297 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:57.671247 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" event={"ID":"9b9575a6-2fca-43e5-b804-1e5e32f9dc68","Type":"ContainerStarted","Data":"c5f47ef8c681e9c64dc2ffc10a391ef33f7257a020f1435d0488d7a7f2e1e7a3"} Jul 20 18:06:57.675864 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:57.675825 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" event={"ID":"94b2cfc1-12b2-4d23-b444-63ccc12c6f06","Type":"ContainerStarted","Data":"a5d5c3411c12ecf94e469a369ef721459639d99af886a1db52d15a36f06dfe81"} Jul 20 18:06:59.691136 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:59.690971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" event={"ID":"9b9575a6-2fca-43e5-b804-1e5e32f9dc68","Type":"ContainerStarted","Data":"f29e0cf425ee45a11a452d76e257f4161839e322dc8bfa0fadae006c55473d96"} Jul 20 18:06:59.691136 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:59.691042 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:06:59.692786 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:59.692756 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" event={"ID":"94b2cfc1-12b2-4d23-b444-63ccc12c6f06","Type":"ContainerStarted","Data":"f6694c8c080dd5c593a9df1f20059d0e315dd28083ace04ddeefbd3cb8a12953"} Jul 20 18:06:59.692917 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:59.692819 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:06:59.709666 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:59.709624 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" podStartSLOduration=1.434850032 podStartE2EDuration="3.709611768s" totalImagesPullingTime="2.274761736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:06:59.707402542 +0000 UTC m=+648.835519707" watchObservedRunningTime="2026-07-20 18:06:59.709611768 +0000 UTC m=+648.837728934" Jul 20 18:06:59.724671 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:06:59.724619 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" podStartSLOduration=1.6780352710000002 podStartE2EDuration="3.724605609s" totalImagesPullingTime="2.046570338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:06:59.722878037 +0000 UTC m=+648.850995229" watchObservedRunningTime="2026-07-20 18:06:59.724605609 +0000 UTC m=+648.852722774" Jul 20 18:07:04.729619 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:04.729533 2567 generic.go:356] "Generic (PLEG): container finished" podID="8c5f0746-dcec-430b-b8ea-9ced3d2a43ca" containerID="3f0117c52a22033fa811283626b1041c157e7bb4ed77e42a092aac14c3943bfa" exitCode=0 Jul 20 18:07:04.729619 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:04.729572 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" event={"ID":"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca","Type":"ContainerDied","Data":"3f0117c52a22033fa811283626b1041c157e7bb4ed77e42a092aac14c3943bfa"} Jul 20 18:07:04.730135 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:04.729931 2567 scope.go:117] "RemoveContainer" containerID="3f0117c52a22033fa811283626b1041c157e7bb4ed77e42a092aac14c3943bfa" Jul 20 18:07:04.730897 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:04.730870 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:07:04.730965 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:04.730936 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:07:04.731009 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:04.730969 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:07:05.737723 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:05.737684 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" event={"ID":"8c5f0746-dcec-430b-b8ea-9ced3d2a43ca","Type":"ContainerStarted","Data":"262f96af2bab5d081e8afc8a83a0ea9d57cdb2f1c5382328ccb565683629f21f"} Jul 20 18:07:06.291132 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:06.291096 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:07:06.292206 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:06.292159 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m9ntd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 20 18:07:06.292312 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:06.292232 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" podUID="8c5f0746-dcec-430b-b8ea-9ced3d2a43ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 20 18:07:07.291531 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:07.291495 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m9ntd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 20 18:07:07.291909 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:07.291554 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" podUID="8c5f0746-dcec-430b-b8ea-9ced3d2a43ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 20 18:07:08.292148 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:08.292095 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m9ntd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 20 18:07:08.292572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:08.292170 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" podUID="8c5f0746-dcec-430b-b8ea-9ced3d2a43ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 20 18:07:09.291987 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:09.291944 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m9ntd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 20 18:07:09.292215 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:09.292004 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" podUID="8c5f0746-dcec-430b-b8ea-9ced3d2a43ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 20 18:07:10.154527 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:10.154486 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:07:10.295157 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:10.295128 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:07:10.701692 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:10.701661 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wp2ks" Jul 20 18:07:10.702066 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:10.702048 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-5nm5f" Jul 20 18:07:10.779157 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:10.779129 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m9ntd" Jul 20 18:07:20.138552 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.138516 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv"] Jul 20 18:07:20.142117 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.142090 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.146041 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.146021 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 18:07:20.146275 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.146259 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 20 18:07:20.147007 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.146986 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 18:07:20.147173 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.147156 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4dzgb\"" Jul 20 18:07:20.164874 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.164850 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv"] Jul 20 18:07:20.176736 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.176711 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.176866 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.176753 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67fgb\" (UniqueName: \"kubernetes.io/projected/dfa09ac6-3677-47a8-9215-967b2edacfe0-kube-api-access-67fgb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.176866 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.176788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.176990 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.176911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.176990 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.176943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.176990 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.176977 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa09ac6-3677-47a8-9215-967b2edacfe0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.278026 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.277988 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.278239 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.278035 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67fgb\" (UniqueName: \"kubernetes.io/projected/dfa09ac6-3677-47a8-9215-967b2edacfe0-kube-api-access-67fgb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.278239 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.278056 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.278239 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.278124 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.278239 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.278140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.278239 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.278168 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa09ac6-3677-47a8-9215-967b2edacfe0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.279700 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.279627 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.280109 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.280075 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.280346 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.280325 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.281525 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.281490 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa09ac6-3677-47a8-9215-967b2edacfe0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.281724 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.281708 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa09ac6-3677-47a8-9215-967b2edacfe0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.287284 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.287254 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67fgb\" (UniqueName: \"kubernetes.io/projected/dfa09ac6-3677-47a8-9215-967b2edacfe0-kube-api-access-67fgb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv\" (UID: \"dfa09ac6-3677-47a8-9215-967b2edacfe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.453348 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.453244 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:20.598019 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.597992 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv"] Jul 20 18:07:20.599463 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:07:20.599427 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa09ac6_3677_47a8_9215_967b2edacfe0.slice/crio-bff47974bd95baf6f971a9cc7d0bdbbd6d0bdd874a8f5647a8f8cc51863b3723 WatchSource:0}: Error finding container bff47974bd95baf6f971a9cc7d0bdbbd6d0bdd874a8f5647a8f8cc51863b3723: Status 404 returned error can't find the container with id bff47974bd95baf6f971a9cc7d0bdbbd6d0bdd874a8f5647a8f8cc51863b3723 Jul 20 18:07:20.838727 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:20.838676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" event={"ID":"dfa09ac6-3677-47a8-9215-967b2edacfe0","Type":"ContainerStarted","Data":"bff47974bd95baf6f971a9cc7d0bdbbd6d0bdd874a8f5647a8f8cc51863b3723"} Jul 20 18:07:22.853146 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:22.853105 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" event={"ID":"dfa09ac6-3677-47a8-9215-967b2edacfe0","Type":"ContainerStarted","Data":"5289c28fead05fc7ba18f7be9b2b3c5f6bd040be62c4e87ad80770ff92a9e79c"} Jul 20 18:07:22.881269 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:22.881219 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" podStartSLOduration=1.5784354569999999 podStartE2EDuration="2.881203084s" totalImagesPullingTime="1.302767627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:07:22.878989887 +0000 UTC m=+672.007107052" watchObservedRunningTime="2026-07-20 18:07:22.881203084 +0000 UTC m=+672.009320241" Jul 20 18:07:30.454225 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:30.454160 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:30.454653 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:30.454248 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:30.466954 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:30.466918 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:30.913911 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:30.913880 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv" Jul 20 18:07:55.024149 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.024092 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj"] Jul 20 18:07:55.031098 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.031076 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.033670 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.033644 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 20 18:07:55.036131 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.036105 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj"] Jul 20 18:07:55.107365 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.107327 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.107526 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.107437 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc13a29-690b-41af-bb91-4c4f3758f840-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.107526 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.107497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.107609 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.107537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.107609 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.107580 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.107675 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.107629 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhvlf\" (UniqueName: \"kubernetes.io/projected/fbc13a29-690b-41af-bb91-4c4f3758f840-kube-api-access-vhvlf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.208828 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.208794 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.208842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc13a29-690b-41af-bb91-4c4f3758f840-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.208865 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.208902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.208921 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209028 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.208965 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvlf\" (UniqueName: \"kubernetes.io/projected/fbc13a29-690b-41af-bb91-4c4f3758f840-kube-api-access-vhvlf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209355 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.209302 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209355 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.209305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.209355 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.209335 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.211171 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.211150 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbc13a29-690b-41af-bb91-4c4f3758f840-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.211498 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.211480 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc13a29-690b-41af-bb91-4c4f3758f840-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.217290 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.217268 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhvlf\" (UniqueName: \"kubernetes.io/projected/fbc13a29-690b-41af-bb91-4c4f3758f840-kube-api-access-vhvlf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj\" (UID: \"fbc13a29-690b-41af-bb91-4c4f3758f840\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.343681 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.343635 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:07:55.494808 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:55.494776 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj"] Jul 20 18:07:55.496042 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:07:55.496017 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc13a29_690b_41af_bb91_4c4f3758f840.slice/crio-35e3ddd36a9139205de682b2827deec602294dcdb8fe140a592775a1b0517847 WatchSource:0}: Error finding container 35e3ddd36a9139205de682b2827deec602294dcdb8fe140a592775a1b0517847: Status 404 returned error can't find the container with id 35e3ddd36a9139205de682b2827deec602294dcdb8fe140a592775a1b0517847 Jul 20 18:07:56.053684 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:56.053633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" event={"ID":"fbc13a29-690b-41af-bb91-4c4f3758f840","Type":"ContainerStarted","Data":"13d5c5d88a89672751757d0d2f5e6e74d00518aa8dde01ac691be0380a7305a9"} Jul 20 18:07:56.054069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:56.053710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" event={"ID":"fbc13a29-690b-41af-bb91-4c4f3758f840","Type":"ContainerStarted","Data":"35e3ddd36a9139205de682b2827deec602294dcdb8fe140a592775a1b0517847"} Jul 20 18:07:56.071913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:07:56.071746 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" podStartSLOduration=1.858301642 podStartE2EDuration="2.071727042s" totalImagesPullingTime="213.4254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:07:56.07114005 +0000 UTC m=+705.199257232" watchObservedRunningTime="2026-07-20 18:07:56.071727042 +0000 UTC m=+705.199844209" Jul 20 18:08:05.344684 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:08:05.344583 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:08:05.344684 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:08:05.344658 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:08:05.357568 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:08:05.357537 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:08:06.123439 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:08:06.123410 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj" Jul 20 18:09:24.292617 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.292575 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9f5557f5d-28kvz"] Jul 20 18:09:24.298295 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.298266 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.303643 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.303615 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9f5557f5d-28kvz"] Jul 20 18:09:24.371656 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.371623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8fbw\" (UniqueName: \"kubernetes.io/projected/f4415671-9428-4b45-8ba9-5092769a1ccb-kube-api-access-g8fbw\") pod \"authorino-9f5557f5d-28kvz\" (UID: \"f4415671-9428-4b45-8ba9-5092769a1ccb\") " pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.371820 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.371699 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4415671-9428-4b45-8ba9-5092769a1ccb-tls-cert\") pod \"authorino-9f5557f5d-28kvz\" (UID: \"f4415671-9428-4b45-8ba9-5092769a1ccb\") " pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.472548 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.472513 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4415671-9428-4b45-8ba9-5092769a1ccb-tls-cert\") pod \"authorino-9f5557f5d-28kvz\" (UID: \"f4415671-9428-4b45-8ba9-5092769a1ccb\") " pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.472721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.472632 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8fbw\" (UniqueName: \"kubernetes.io/projected/f4415671-9428-4b45-8ba9-5092769a1ccb-kube-api-access-g8fbw\") pod \"authorino-9f5557f5d-28kvz\" (UID: \"f4415671-9428-4b45-8ba9-5092769a1ccb\") " pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.475228 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.475168 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4415671-9428-4b45-8ba9-5092769a1ccb-tls-cert\") pod \"authorino-9f5557f5d-28kvz\" (UID: \"f4415671-9428-4b45-8ba9-5092769a1ccb\") " pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.482233 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.482209 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8fbw\" (UniqueName: \"kubernetes.io/projected/f4415671-9428-4b45-8ba9-5092769a1ccb-kube-api-access-g8fbw\") pod \"authorino-9f5557f5d-28kvz\" (UID: \"f4415671-9428-4b45-8ba9-5092769a1ccb\") " pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.609550 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.609507 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9f5557f5d-28kvz" Jul 20 18:09:24.756895 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:24.756865 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9f5557f5d-28kvz"] Jul 20 18:09:24.758866 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:09:24.758833 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4415671_9428_4b45_8ba9_5092769a1ccb.slice/crio-6772e8820cb069078bcc2f4bfa071c128b9125dd191db6fc3ba44977553bcd71 WatchSource:0}: Error finding container 6772e8820cb069078bcc2f4bfa071c128b9125dd191db6fc3ba44977553bcd71: Status 404 returned error can't find the container with id 6772e8820cb069078bcc2f4bfa071c128b9125dd191db6fc3ba44977553bcd71 Jul 20 18:09:25.591498 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.591465 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9f5557f5d-28kvz" event={"ID":"f4415671-9428-4b45-8ba9-5092769a1ccb","Type":"ContainerStarted","Data":"53c0a8798321d79025a7120cb831e2e19a843ff3694ae8c4d2924547a0f43193"} Jul 20 18:09:25.591908 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.591509 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9f5557f5d-28kvz" event={"ID":"f4415671-9428-4b45-8ba9-5092769a1ccb","Type":"ContainerStarted","Data":"6772e8820cb069078bcc2f4bfa071c128b9125dd191db6fc3ba44977553bcd71"} Jul 20 18:09:25.613133 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.613065 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9f5557f5d-28kvz" podStartSLOduration=1.208326467 podStartE2EDuration="1.613046885s" totalImagesPullingTime="404.720418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:09:25.610470247 +0000 UTC m=+794.738587414" watchObservedRunningTime="2026-07-20 18:09:25.613046885 +0000 UTC m=+794.741164051" Jul 20 18:09:25.642255 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.642208 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d56b7dfdc-djsn4"] Jul 20 18:09:25.642499 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.642471 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" podUID="bfa41074-8909-43fa-bf62-a1863ca43d6d" containerName="authorino" containerID="cri-o://29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5" gracePeriod=30 Jul 20 18:09:25.927206 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.927169 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:09:25.992283 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.992246 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq9zq\" (UniqueName: \"kubernetes.io/projected/bfa41074-8909-43fa-bf62-a1863ca43d6d-kube-api-access-xq9zq\") pod \"bfa41074-8909-43fa-bf62-a1863ca43d6d\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " Jul 20 18:09:25.992462 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.992308 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa41074-8909-43fa-bf62-a1863ca43d6d-tls-cert\") pod \"bfa41074-8909-43fa-bf62-a1863ca43d6d\" (UID: \"bfa41074-8909-43fa-bf62-a1863ca43d6d\") " Jul 20 18:09:25.994533 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:25.994489 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfa41074-8909-43fa-bf62-a1863ca43d6d-kube-api-access-xq9zq" (OuterVolumeSpecName: "kube-api-access-xq9zq") pod "bfa41074-8909-43fa-bf62-a1863ca43d6d" (UID: "bfa41074-8909-43fa-bf62-a1863ca43d6d"). InnerVolumeSpecName "kube-api-access-xq9zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:09:26.006494 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.006459 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfa41074-8909-43fa-bf62-a1863ca43d6d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "bfa41074-8909-43fa-bf62-a1863ca43d6d" (UID: "bfa41074-8909-43fa-bf62-a1863ca43d6d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:09:26.093633 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.093595 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq9zq\" (UniqueName: \"kubernetes.io/projected/bfa41074-8909-43fa-bf62-a1863ca43d6d-kube-api-access-xq9zq\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:09:26.093633 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.093624 2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa41074-8909-43fa-bf62-a1863ca43d6d-tls-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:09:26.600370 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.600334 2567 generic.go:356] "Generic (PLEG): container finished" podID="bfa41074-8909-43fa-bf62-a1863ca43d6d" containerID="29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5" exitCode=0 Jul 20 18:09:26.600849 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.600398 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" Jul 20 18:09:26.600849 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.600417 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" event={"ID":"bfa41074-8909-43fa-bf62-a1863ca43d6d","Type":"ContainerDied","Data":"29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5"} Jul 20 18:09:26.600849 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.600467 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d56b7dfdc-djsn4" event={"ID":"bfa41074-8909-43fa-bf62-a1863ca43d6d","Type":"ContainerDied","Data":"6bcab814b13b2914882083a0b2391fdb83c322b0a2fb6398c784787388799d00"} Jul 20 18:09:26.600849 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.600492 2567 scope.go:117] "RemoveContainer" containerID="29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5" Jul 20 18:09:26.611220 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.611165 2567 scope.go:117] "RemoveContainer" containerID="29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5" Jul 20 18:09:26.611565 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:09:26.611542 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5\": container with ID starting with 29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5 not found: ID does not exist" containerID="29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5" Jul 20 18:09:26.611634 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.611576 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5"} err="failed to get container status \"29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5\": rpc error: code = NotFound desc = could not find container \"29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5\": container with ID starting with 29d66257dbfe38dbb38a37f16b526d23beef3d9fb3894ba865e535e4c46418a5 not found: ID does not exist" Jul 20 18:09:26.627798 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.627759 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d56b7dfdc-djsn4"] Jul 20 18:09:26.631611 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:26.631583 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d56b7dfdc-djsn4"] Jul 20 18:09:27.442165 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:09:27.442123 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfa41074-8909-43fa-bf62-a1863ca43d6d" path="/var/lib/kubelet/pods/bfa41074-8909-43fa-bf62-a1863ca43d6d/volumes" Jul 20 18:10:13.575663 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.575623 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bd97c8594-bcxw9"] Jul 20 18:10:13.576145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.575893 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" podUID="8f16886b-e348-4d7e-ad04-7ef070593d3a" containerName="manager" containerID="cri-o://c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad" gracePeriod=30 Jul 20 18:10:13.848537 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.848508 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:10:13.894526 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.894491 2567 generic.go:356] "Generic (PLEG): container finished" podID="8f16886b-e348-4d7e-ad04-7ef070593d3a" containerID="c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad" exitCode=0 Jul 20 18:10:13.894721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.894545 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" Jul 20 18:10:13.894721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.894558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" event={"ID":"8f16886b-e348-4d7e-ad04-7ef070593d3a","Type":"ContainerDied","Data":"c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad"} Jul 20 18:10:13.894721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.894614 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd97c8594-bcxw9" event={"ID":"8f16886b-e348-4d7e-ad04-7ef070593d3a","Type":"ContainerDied","Data":"d0eedc265a7d398d4c0cfd5283363c2818e57289dd53979708baea1d4cea72ad"} Jul 20 18:10:13.894721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.894648 2567 scope.go:117] "RemoveContainer" containerID="c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad" Jul 20 18:10:13.906657 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.906634 2567 scope.go:117] "RemoveContainer" containerID="c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad" Jul 20 18:10:13.907038 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:10:13.907016 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad\": container with ID starting with c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad not found: ID does not exist" containerID="c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad" Jul 20 18:10:13.907122 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.907054 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad"} err="failed to get container status \"c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad\": rpc error: code = NotFound desc = could not find container \"c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad\": container with ID starting with c4f19b2f01044b8e5be8ef0677fc0ed8360619b254dac56215b0491d566d45ad not found: ID does not exist" Jul 20 18:10:13.957292 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.957258 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f16886b-e348-4d7e-ad04-7ef070593d3a-webhook-cert\") pod \"8f16886b-e348-4d7e-ad04-7ef070593d3a\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " Jul 20 18:10:13.957474 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.957370 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh7qb\" (UniqueName: \"kubernetes.io/projected/8f16886b-e348-4d7e-ad04-7ef070593d3a-kube-api-access-zh7qb\") pod \"8f16886b-e348-4d7e-ad04-7ef070593d3a\" (UID: \"8f16886b-e348-4d7e-ad04-7ef070593d3a\") " Jul 20 18:10:13.959586 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.959550 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f16886b-e348-4d7e-ad04-7ef070593d3a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8f16886b-e348-4d7e-ad04-7ef070593d3a" (UID: "8f16886b-e348-4d7e-ad04-7ef070593d3a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:10:13.959716 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:13.959587 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f16886b-e348-4d7e-ad04-7ef070593d3a-kube-api-access-zh7qb" (OuterVolumeSpecName: "kube-api-access-zh7qb") pod "8f16886b-e348-4d7e-ad04-7ef070593d3a" (UID: "8f16886b-e348-4d7e-ad04-7ef070593d3a"). InnerVolumeSpecName "kube-api-access-zh7qb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:10:14.059081 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:14.059050 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f16886b-e348-4d7e-ad04-7ef070593d3a-webhook-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:10:14.059081 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:14.059079 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh7qb\" (UniqueName: \"kubernetes.io/projected/8f16886b-e348-4d7e-ad04-7ef070593d3a-kube-api-access-zh7qb\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:10:14.218942 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:14.218902 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bd97c8594-bcxw9"] Jul 20 18:10:14.223467 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:14.223432 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bd97c8594-bcxw9"] Jul 20 18:10:15.441870 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:10:15.441834 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f16886b-e348-4d7e-ad04-7ef070593d3a" path="/var/lib/kubelet/pods/8f16886b-e348-4d7e-ad04-7ef070593d3a/volumes" Jul 20 18:11:11.351889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:11:11.351807 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:11:11.353463 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:11:11.353428 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:16:11.392612 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:16:11.392584 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:16:11.395944 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:16:11.395918 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:21:11.437912 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:21:11.437871 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:21:11.452508 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:21:11.452479 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:24:37.462620 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.462536 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq"] Jul 20 18:24:37.464717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.463198 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f16886b-e348-4d7e-ad04-7ef070593d3a" containerName="manager" Jul 20 18:24:37.464717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.463218 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f16886b-e348-4d7e-ad04-7ef070593d3a" containerName="manager" Jul 20 18:24:37.464717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.463228 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfa41074-8909-43fa-bf62-a1863ca43d6d" containerName="authorino" Jul 20 18:24:37.464717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.463233 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfa41074-8909-43fa-bf62-a1863ca43d6d" containerName="authorino" Jul 20 18:24:37.464717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.463302 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f16886b-e348-4d7e-ad04-7ef070593d3a" containerName="manager" Jul 20 18:24:37.464717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.463312 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfa41074-8909-43fa-bf62-a1863ca43d6d" containerName="authorino" Jul 20 18:24:37.466357 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.466341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.469477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.469456 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7f0c0a5b-openshift-default-dockercfg-46bhg\"" Jul 20 18:24:37.521001 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.520962 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e7b68c-6c94-496f-ae80-170548ace6e5-istiod-ca-cert\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521176 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klgq9\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-kube-api-access-klgq9\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521176 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521054 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-socket\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521176 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-credential-socket\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521176 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-data\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521176 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-token\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521431 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521213 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-envoy\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521431 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521229 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-podinfo\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.521431 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.521297 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-certs\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.530445 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.530414 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq"] Jul 20 18:24:37.621853 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.621817 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e7b68c-6c94-496f-ae80-170548ace6e5-istiod-ca-cert\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622044 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.621870 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klgq9\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-kube-api-access-klgq9\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622044 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.621985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-socket\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622044 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622030 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-credential-socket\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622067 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-data\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622093 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-token\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622129 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-envoy\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-podinfo\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-certs\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622535 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622497 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-socket\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622626 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622599 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-data\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622707 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622650 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-certs\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622707 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622687 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e7b68c-6c94-496f-ae80-170548ace6e5-istiod-ca-cert\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.622807 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.622762 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-credential-socket\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.624534 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.624511 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-envoy\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.624728 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.624710 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-podinfo\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.631362 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.631333 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klgq9\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-kube-api-access-klgq9\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.631483 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.631466 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-token\") pod \"e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.776956 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.776862 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:37.931152 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.931115 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq"] Jul 20 18:24:37.932482 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:24:37.932458 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e7b68c_6c94_496f_ae80_170548ace6e5.slice/crio-237cec24548679976f64116d3967b071c6a9416ee07be482a8376c0bc561ff3e WatchSource:0}: Error finding container 237cec24548679976f64116d3967b071c6a9416ee07be482a8376c0bc561ff3e: Status 404 returned error can't find the container with id 237cec24548679976f64116d3967b071c6a9416ee07be482a8376c0bc561ff3e Jul 20 18:24:37.934676 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.934647 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:24:37.934754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.934711 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:24:37.934754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:37.934737 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:24:38.112335 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:38.112302 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" event={"ID":"23e7b68c-6c94-496f-ae80-170548ace6e5","Type":"ContainerStarted","Data":"4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480"} Jul 20 18:24:38.112504 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:38.112356 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" event={"ID":"23e7b68c-6c94-496f-ae80-170548ace6e5","Type":"ContainerStarted","Data":"237cec24548679976f64116d3967b071c6a9416ee07be482a8376c0bc561ff3e"} Jul 20 18:24:38.134124 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:38.134071 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podStartSLOduration=1.134055454 podStartE2EDuration="1.134055454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:24:38.133119086 +0000 UTC m=+1707.261236277" watchObservedRunningTime="2026-07-20 18:24:38.134055454 +0000 UTC m=+1707.262172620" Jul 20 18:24:38.777927 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:38.777893 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:38.782781 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:38.782752 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:39.118070 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:39.118041 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:39.119102 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:39.119080 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:55.159352 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:55.159312 2567 patch_prober.go:28] interesting pod/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 18:24:55.159750 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:55.159377 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 18:24:56.227514 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:56.227469 2567 generic.go:356] "Generic (PLEG): container finished" podID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerID="4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480" exitCode=0 Jul 20 18:24:56.228039 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:56.227566 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" event={"ID":"23e7b68c-6c94-496f-ae80-170548ace6e5","Type":"ContainerDied","Data":"4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480"} Jul 20 18:24:56.228495 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:56.228309 2567 scope.go:117] "RemoveContainer" containerID="4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480" Jul 20 18:24:56.232196 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:56.232139 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:24:56.232396 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:56.232372 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:24:56.232591 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:56.232568 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:24:57.236997 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:57.236968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" event={"ID":"23e7b68c-6c94-496f-ae80-170548ace6e5","Type":"ContainerStarted","Data":"cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f"} Jul 20 18:24:57.777044 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:57.777008 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:24:57.777287 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:57.777226 2567 patch_prober.go:28] interesting pod/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 20 18:24:57.777372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:57.777283 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 20 18:24:58.778159 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:58.778122 2567 patch_prober.go:28] interesting pod/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 20 18:24:58.778672 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:58.778215 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 20 18:24:59.777324 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:59.777279 2567 patch_prober.go:28] interesting pod/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 20 18:24:59.777492 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:24:59.777346 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 20 18:25:00.777474 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:00.777431 2567 patch_prober.go:28] interesting pod/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 20 18:25:00.777860 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:00.777497 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 20 18:25:01.781278 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:01.781235 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:25:01.781741 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:01.781712 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:25:01.782464 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:01.782446 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:25:20.418401 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:20.418367 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq"] Jul 20 18:25:20.418917 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:20.418595 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" containerID="cri-o://cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f" gracePeriod=30 Jul 20 18:25:25.677893 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.677866 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:25:25.707326 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707237 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-token\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707326 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707278 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-podinfo\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707326 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707308 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-envoy\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707387 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e7b68c-6c94-496f-ae80-170548ace6e5-istiod-ca-cert\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707413 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-credential-socket\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707439 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-certs\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707484 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-data\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707542 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-socket\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707589 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klgq9\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-kube-api-access-klgq9\") pod \"23e7b68c-6c94-496f-ae80-170548ace6e5\" (UID: \"23e7b68c-6c94-496f-ae80-170548ace6e5\") " Jul 20 18:25:25.707913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707746 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:25:25.707970 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.707920 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:25:25.708068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.708049 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-credential-socket\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.708132 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.708074 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-certs\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.708132 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.708092 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:25:25.708418 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.708393 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e7b68c-6c94-496f-ae80-170548ace6e5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:25:25.708626 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.708601 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-data" (OuterVolumeSpecName: "istio-data") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:25:25.709885 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.709857 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:25:25.710109 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.710084 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:25:25.710221 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.710173 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 18:25:25.710338 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.710319 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-kube-api-access-klgq9" (OuterVolumeSpecName: "kube-api-access-klgq9") pod "23e7b68c-6c94-496f-ae80-170548ace6e5" (UID: "23e7b68c-6c94-496f-ae80-170548ace6e5"). InnerVolumeSpecName "kube-api-access-klgq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:25:25.809517 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809479 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-data\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.809517 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809510 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-workload-socket\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.809517 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809520 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klgq9\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-kube-api-access-klgq9\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.809517 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809529 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-token\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.809793 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809537 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-podinfo\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.809793 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809545 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23e7b68c-6c94-496f-ae80-170548ace6e5-istio-envoy\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:25.809793 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:25.809554 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23e7b68c-6c94-496f-ae80-170548ace6e5-istiod-ca-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:25:26.427769 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.427731 2567 generic.go:356] "Generic (PLEG): container finished" podID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerID="cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f" exitCode=0 Jul 20 18:25:26.427975 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.427805 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" Jul 20 18:25:26.427975 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.427822 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" event={"ID":"23e7b68c-6c94-496f-ae80-170548ace6e5","Type":"ContainerDied","Data":"cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f"} Jul 20 18:25:26.427975 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.427907 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq" event={"ID":"23e7b68c-6c94-496f-ae80-170548ace6e5","Type":"ContainerDied","Data":"237cec24548679976f64116d3967b071c6a9416ee07be482a8376c0bc561ff3e"} Jul 20 18:25:26.427975 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.427932 2567 scope.go:117] "RemoveContainer" containerID="cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f" Jul 20 18:25:26.438384 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.438362 2567 scope.go:117] "RemoveContainer" containerID="4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480" Jul 20 18:25:26.446761 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.446738 2567 scope.go:117] "RemoveContainer" containerID="cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f" Jul 20 18:25:26.447005 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:25:26.446983 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f\": container with ID starting with cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f not found: ID does not exist" containerID="cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f" Jul 20 18:25:26.447068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.447013 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f"} err="failed to get container status \"cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f\": rpc error: code = NotFound desc = could not find container \"cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f\": container with ID starting with cc285f58e1806605ea0c4acc642e4d6d7761d04afb4f729d12ae7de14884885f not found: ID does not exist" Jul 20 18:25:26.447068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.447032 2567 scope.go:117] "RemoveContainer" containerID="4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480" Jul 20 18:25:26.447311 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:25:26.447294 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480\": container with ID starting with 4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480 not found: ID does not exist" containerID="4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480" Jul 20 18:25:26.447363 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.447317 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480"} err="failed to get container status \"4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480\": rpc error: code = NotFound desc = could not find container \"4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480\": container with ID starting with 4937400b6faf80aed15f0f972c03d0fc5277d11892decd32e795adfbfd064480 not found: ID does not exist" Jul 20 18:25:26.453709 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.453685 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq"] Jul 20 18:25:26.458498 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.458471 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7f0c0a5b-openshift-default-7866b5b895-bw9mq"] Jul 20 18:25:26.629449 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629408 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l"] Jul 20 18:25:26.629870 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629857 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" Jul 20 18:25:26.629913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629872 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" Jul 20 18:25:26.629913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629889 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" Jul 20 18:25:26.629913 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629895 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" Jul 20 18:25:26.630010 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629968 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" Jul 20 18:25:26.630010 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.629980 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" containerName="istio-proxy" Jul 20 18:25:26.633597 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.633578 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.637759 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.637738 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h2tfv\"" Jul 20 18:25:26.637891 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.637778 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0ae63acc-serving-cert\"" Jul 20 18:25:26.646571 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.646547 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l"] Jul 20 18:25:26.725909 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.725814 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.725909 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.725876 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-openshift-service-ca\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.726367 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.725962 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfj68\" (UniqueName: \"kubernetes.io/projected/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-kube-api-access-wfj68\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.826961 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.826919 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.827199 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.826977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-openshift-service-ca\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.827199 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.827022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfj68\" (UniqueName: \"kubernetes.io/projected/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-kube-api-access-wfj68\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.827199 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:25:26.827078 2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-serving-cert: secret "maas-api-e2e-migrate-0ae63acc-serving-cert" not found Jul 20 18:25:26.827199 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:25:26.827159 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls podName:af7fadfb-d3e4-4727-b2c0-43dd0614f2b6 nodeName:}" failed. No retries permitted until 2026-07-20 18:25:27.327142285 +0000 UTC m=+1756.455259447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls") pod "maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" (UID: "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6") : secret "maas-api-e2e-migrate-0ae63acc-serving-cert" not found Jul 20 18:25:26.827645 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.827624 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-openshift-service-ca\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:26.837882 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:26.837850 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfj68\" (UniqueName: \"kubernetes.io/projected/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-kube-api-access-wfj68\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:27.331400 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:27.331365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:27.333780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:27.333760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls\") pod \"maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:27.441653 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:27.441620 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23e7b68c-6c94-496f-ae80-170548ace6e5" path="/var/lib/kubelet/pods/23e7b68c-6c94-496f-ae80-170548ace6e5/volumes" Jul 20 18:25:27.545094 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:27.545059 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:27.709336 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:27.709309 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l"] Jul 20 18:25:27.711261 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:25:27.711234 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7fadfb_d3e4_4727_b2c0_43dd0614f2b6.slice/crio-c67782de55137c91f847a6d0525d6d23495842c4383579ee2cde9fc5a45f9fea WatchSource:0}: Error finding container c67782de55137c91f847a6d0525d6d23495842c4383579ee2cde9fc5a45f9fea: Status 404 returned error can't find the container with id c67782de55137c91f847a6d0525d6d23495842c4383579ee2cde9fc5a45f9fea Jul 20 18:25:27.712575 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:27.712556 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 18:25:28.444708 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:28.444672 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" event={"ID":"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6","Type":"ContainerStarted","Data":"c67782de55137c91f847a6d0525d6d23495842c4383579ee2cde9fc5a45f9fea"} Jul 20 18:25:30.458219 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:30.458163 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" event={"ID":"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6","Type":"ContainerStarted","Data":"31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143"} Jul 20 18:25:30.458597 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:30.458242 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:25:30.478324 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:30.478266 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" podStartSLOduration=2.709202851 podStartE2EDuration="4.478246613s" totalImagesPullingTime="1.769043762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:25:30.475704993 +0000 UTC m=+1759.603822187" watchObservedRunningTime="2026-07-20 18:25:30.478246613 +0000 UTC m=+1759.606363778" Jul 20 18:25:36.469151 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:25:36.469119 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:26:01.963428 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:01.963379 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l"] Jul 20 18:26:01.964012 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:01.963673 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" podUID="af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" containerName="maas-api" containerID="cri-o://31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143" gracePeriod=30 Jul 20 18:26:02.225458 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.225392 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:26:02.271635 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.271596 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfj68\" (UniqueName: \"kubernetes.io/projected/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-kube-api-access-wfj68\") pod \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " Jul 20 18:26:02.271825 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.271676 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-openshift-service-ca\") pod \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " Jul 20 18:26:02.271825 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.271801 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls\") pod \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\" (UID: \"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6\") " Jul 20 18:26:02.272080 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.272056 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" (UID: "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:26:02.272168 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.272122 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-openshift-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:26:02.273917 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.273884 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" (UID: "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:26:02.274009 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.273931 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-kube-api-access-wfj68" (OuterVolumeSpecName: "kube-api-access-wfj68") pod "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" (UID: "af7fadfb-d3e4-4727-b2c0-43dd0614f2b6"). InnerVolumeSpecName "kube-api-access-wfj68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:26:02.373569 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.373531 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-maas-api-tls\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:26:02.373569 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.373564 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfj68\" (UniqueName: \"kubernetes.io/projected/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6-kube-api-access-wfj68\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:26:02.648291 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.648252 2567 generic.go:356] "Generic (PLEG): container finished" podID="af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" containerID="31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143" exitCode=0 Jul 20 18:26:02.648291 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.648288 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" Jul 20 18:26:02.648491 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.648305 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" event={"ID":"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6","Type":"ContainerDied","Data":"31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143"} Jul 20 18:26:02.648491 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.648364 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l" event={"ID":"af7fadfb-d3e4-4727-b2c0-43dd0614f2b6","Type":"ContainerDied","Data":"c67782de55137c91f847a6d0525d6d23495842c4383579ee2cde9fc5a45f9fea"} Jul 20 18:26:02.648491 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.648393 2567 scope.go:117] "RemoveContainer" containerID="31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143" Jul 20 18:26:02.658453 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.658431 2567 scope.go:117] "RemoveContainer" containerID="31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143" Jul 20 18:26:02.658732 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:26:02.658708 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143\": container with ID starting with 31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143 not found: ID does not exist" containerID="31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143" Jul 20 18:26:02.658822 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.658746 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143"} err="failed to get container status \"31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143\": rpc error: code = NotFound desc = could not find container \"31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143\": container with ID starting with 31f6751ba6d1d1831f2d8abaf8166f0da01da790c4857be146ed68f957258143 not found: ID does not exist" Jul 20 18:26:02.671737 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.671699 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l"] Jul 20 18:26:02.685169 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:02.685135 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0ae63acc-6c594f9fd6-5ms8l"] Jul 20 18:26:03.441057 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:03.441021 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" path="/var/lib/kubelet/pods/af7fadfb-d3e4-4727-b2c0-43dd0614f2b6/volumes" Jul 20 18:26:09.602788 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.602760 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5"] Jul 20 18:26:09.603633 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.603608 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" containerName="maas-api" Jul 20 18:26:09.603633 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.603631 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" containerName="maas-api" Jul 20 18:26:09.603789 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.603720 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="af7fadfb-d3e4-4727-b2c0-43dd0614f2b6" containerName="maas-api" Jul 20 18:26:09.605906 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.605890 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.608447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.608427 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-674433c6-serving-cert\"" Jul 20 18:26:09.609898 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.609880 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h2tfv\"" Jul 20 18:26:09.622020 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.621993 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5"] Jul 20 18:26:09.752826 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.752778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5tl2\" (UniqueName: \"kubernetes.io/projected/e97a82a9-d550-4252-95b1-46a847821f18-kube-api-access-x5tl2\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.753004 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.752889 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e97a82a9-d550-4252-95b1-46a847821f18-maas-api-tls\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.753004 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.752916 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e97a82a9-d550-4252-95b1-46a847821f18-openshift-service-ca\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.854049 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.853959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e97a82a9-d550-4252-95b1-46a847821f18-maas-api-tls\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.854049 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.853998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e97a82a9-d550-4252-95b1-46a847821f18-openshift-service-ca\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.854270 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.854092 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5tl2\" (UniqueName: \"kubernetes.io/projected/e97a82a9-d550-4252-95b1-46a847821f18-kube-api-access-x5tl2\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.854768 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.854738 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e97a82a9-d550-4252-95b1-46a847821f18-openshift-service-ca\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.856557 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.856529 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e97a82a9-d550-4252-95b1-46a847821f18-maas-api-tls\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.866543 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.866518 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5tl2\" (UniqueName: \"kubernetes.io/projected/e97a82a9-d550-4252-95b1-46a847821f18-kube-api-access-x5tl2\") pod \"maas-api-e2e-ait-674433c6-757d8bb557-c7wx5\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:09.916747 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:09.916710 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:10.063801 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:10.063768 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5"] Jul 20 18:26:10.065152 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:26:10.065123 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97a82a9_d550_4252_95b1_46a847821f18.slice/crio-167bd35cd73f0010638242c7cdc1a332f58f2fe459aae34cff3c971313192d44 WatchSource:0}: Error finding container 167bd35cd73f0010638242c7cdc1a332f58f2fe459aae34cff3c971313192d44: Status 404 returned error can't find the container with id 167bd35cd73f0010638242c7cdc1a332f58f2fe459aae34cff3c971313192d44 Jul 20 18:26:10.706313 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:10.706275 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" event={"ID":"e97a82a9-d550-4252-95b1-46a847821f18","Type":"ContainerStarted","Data":"83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b"} Jul 20 18:26:10.706705 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:10.706339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" event={"ID":"e97a82a9-d550-4252-95b1-46a847821f18","Type":"ContainerStarted","Data":"167bd35cd73f0010638242c7cdc1a332f58f2fe459aae34cff3c971313192d44"} Jul 20 18:26:10.706705 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:10.706369 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:10.727740 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:10.727684 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" podStartSLOduration=1.277861544 podStartE2EDuration="1.727666237s" totalImagesPullingTime="449.804693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:10.726956229 +0000 UTC m=+1799.855073386" watchObservedRunningTime="2026-07-20 18:26:10.727666237 +0000 UTC m=+1799.855783403" Jul 20 18:26:11.488197 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:11.488158 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:26:11.494623 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:11.494601 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:26:16.718040 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:16.718005 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:46.020323 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.020289 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5"] Jul 20 18:26:46.020871 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.020499 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" podUID="e97a82a9-d550-4252-95b1-46a847821f18" containerName="maas-api" containerID="cri-o://83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b" gracePeriod=30 Jul 20 18:26:46.286426 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.286359 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:46.406238 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.406165 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e97a82a9-d550-4252-95b1-46a847821f18-maas-api-tls\") pod \"e97a82a9-d550-4252-95b1-46a847821f18\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " Jul 20 18:26:46.406397 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.406267 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e97a82a9-d550-4252-95b1-46a847821f18-openshift-service-ca\") pod \"e97a82a9-d550-4252-95b1-46a847821f18\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " Jul 20 18:26:46.406397 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.406352 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5tl2\" (UniqueName: \"kubernetes.io/projected/e97a82a9-d550-4252-95b1-46a847821f18-kube-api-access-x5tl2\") pod \"e97a82a9-d550-4252-95b1-46a847821f18\" (UID: \"e97a82a9-d550-4252-95b1-46a847821f18\") " Jul 20 18:26:46.406636 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.406611 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e97a82a9-d550-4252-95b1-46a847821f18-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e97a82a9-d550-4252-95b1-46a847821f18" (UID: "e97a82a9-d550-4252-95b1-46a847821f18"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:26:46.408439 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.408410 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e97a82a9-d550-4252-95b1-46a847821f18-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e97a82a9-d550-4252-95b1-46a847821f18" (UID: "e97a82a9-d550-4252-95b1-46a847821f18"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:26:46.408548 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.408517 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97a82a9-d550-4252-95b1-46a847821f18-kube-api-access-x5tl2" (OuterVolumeSpecName: "kube-api-access-x5tl2") pod "e97a82a9-d550-4252-95b1-46a847821f18" (UID: "e97a82a9-d550-4252-95b1-46a847821f18"). InnerVolumeSpecName "kube-api-access-x5tl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:26:46.507285 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.507248 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5tl2\" (UniqueName: \"kubernetes.io/projected/e97a82a9-d550-4252-95b1-46a847821f18-kube-api-access-x5tl2\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:26:46.507285 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.507278 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e97a82a9-d550-4252-95b1-46a847821f18-maas-api-tls\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:26:46.507285 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.507289 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e97a82a9-d550-4252-95b1-46a847821f18-openshift-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:26:46.930447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.930405 2567 generic.go:356] "Generic (PLEG): container finished" podID="e97a82a9-d550-4252-95b1-46a847821f18" containerID="83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b" exitCode=0 Jul 20 18:26:46.930447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.930447 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" event={"ID":"e97a82a9-d550-4252-95b1-46a847821f18","Type":"ContainerDied","Data":"83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b"} Jul 20 18:26:46.930717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.930479 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" Jul 20 18:26:46.930717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.930518 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5" event={"ID":"e97a82a9-d550-4252-95b1-46a847821f18","Type":"ContainerDied","Data":"167bd35cd73f0010638242c7cdc1a332f58f2fe459aae34cff3c971313192d44"} Jul 20 18:26:46.930717 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.930541 2567 scope.go:117] "RemoveContainer" containerID="83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b" Jul 20 18:26:46.941198 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.941158 2567 scope.go:117] "RemoveContainer" containerID="83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b" Jul 20 18:26:46.941478 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:26:46.941456 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b\": container with ID starting with 83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b not found: ID does not exist" containerID="83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b" Jul 20 18:26:46.941529 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.941488 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b"} err="failed to get container status \"83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b\": rpc error: code = NotFound desc = could not find container \"83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b\": container with ID starting with 83284ff72b54bbcbb04be181451cc97f800b8de46052b8633029d0d578dfb37b not found: ID does not exist" Jul 20 18:26:46.958120 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.958083 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5"] Jul 20 18:26:46.969030 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:46.969003 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-674433c6-757d8bb557-c7wx5"] Jul 20 18:26:47.441248 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:47.441224 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e97a82a9-d550-4252-95b1-46a847821f18" path="/var/lib/kubelet/pods/e97a82a9-d550-4252-95b1-46a847821f18/volumes" Jul 20 18:26:52.428167 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.428133 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf"] Jul 20 18:26:52.428583 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.428569 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97a82a9-d550-4252-95b1-46a847821f18" containerName="maas-api" Jul 20 18:26:52.428583 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.428584 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97a82a9-d550-4252-95b1-46a847821f18" containerName="maas-api" Jul 20 18:26:52.428675 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.428666 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e97a82a9-d550-4252-95b1-46a847821f18" containerName="maas-api" Jul 20 18:26:52.433262 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.433237 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.440403 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.440380 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-76c00924-openshift-default-dockercfg-bsl92\"" Jul 20 18:26:52.455375 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.455337 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf"] Jul 20 18:26:52.572886 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.572844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-envoy\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.572907 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft2ds\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-kube-api-access-ft2ds\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.572934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-data\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.572966 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-socket\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573068 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.572989 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-credential-socket\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573225 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.573068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-certs\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573225 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.573109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-podinfo\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573225 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.573140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5081902a-edad-4292-bdd8-99990a2fd4b6-istiod-ca-cert\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.573225 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.573167 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-token\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674263 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5081902a-edad-4292-bdd8-99990a2fd4b6-istiod-ca-cert\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674442 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674284 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-token\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674442 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674352 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-envoy\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674442 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674405 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft2ds\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-kube-api-access-ft2ds\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674442 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674429 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-data\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674637 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674472 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-socket\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674637 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674498 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-credential-socket\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674637 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-certs\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674637 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-podinfo\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.674907 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674871 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-credential-socket\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.675039 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-socket\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.675039 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.674969 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-data\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.675160 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.675054 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5081902a-edad-4292-bdd8-99990a2fd4b6-istiod-ca-cert\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.675160 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.675104 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-certs\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.676719 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.676696 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-envoy\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.676931 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.676916 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-podinfo\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.685824 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.685762 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-token\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.685917 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.685884 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft2ds\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-kube-api-access-ft2ds\") pod \"e2e-derive-76c00924-openshift-default-fbc677687-clvkf\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.744467 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.744429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:52.898164 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.898131 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf"] Jul 20 18:26:52.899624 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:26:52.899596 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5081902a_edad_4292_bdd8_99990a2fd4b6.slice/crio-68a3e073814f121f658cb71991923804d17519e4695319f94e86d8b14246bbbf WatchSource:0}: Error finding container 68a3e073814f121f658cb71991923804d17519e4695319f94e86d8b14246bbbf: Status 404 returned error can't find the container with id 68a3e073814f121f658cb71991923804d17519e4695319f94e86d8b14246bbbf Jul 20 18:26:52.901749 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.901719 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:26:52.901832 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.901783 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:26:52.901832 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.901811 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:26:52.980923 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.980884 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" event={"ID":"5081902a-edad-4292-bdd8-99990a2fd4b6","Type":"ContainerStarted","Data":"f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213"} Jul 20 18:26:52.981124 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:52.980974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" event={"ID":"5081902a-edad-4292-bdd8-99990a2fd4b6","Type":"ContainerStarted","Data":"68a3e073814f121f658cb71991923804d17519e4695319f94e86d8b14246bbbf"} Jul 20 18:26:53.016653 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:53.016595 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" podStartSLOduration=1.016575898 podStartE2EDuration="1.016575898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:53.01336762 +0000 UTC m=+1842.141484786" watchObservedRunningTime="2026-07-20 18:26:53.016575898 +0000 UTC m=+1842.144693141" Jul 20 18:26:53.744786 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:53.744747 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:53.749849 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:53.749822 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:53.987256 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:53.987224 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:26:53.988063 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:26:53.988041 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:08.057063 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:27:08.057019 2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5081902a_edad_4292_bdd8_99990a2fd4b6.slice/crio-f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213.scope\": RecentStats: unable to find data in memory cache], [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@2-27697-0.service\": RecentStats: unable to find data in memory cache]" Jul 20 18:27:08.079860 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.079826 2567 generic.go:356] "Generic (PLEG): container finished" podID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerID="f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213" exitCode=0 Jul 20 18:27:08.080010 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.079895 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" event={"ID":"5081902a-edad-4292-bdd8-99990a2fd4b6","Type":"ContainerDied","Data":"f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213"} Jul 20 18:27:08.080320 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.080307 2567 scope.go:117] "RemoveContainer" containerID="f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213" Jul 20 18:27:08.081382 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.081353 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:27:08.081503 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.081449 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:27:08.081572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.081500 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:27:08.987903 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:08.987855 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:09.088326 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:09.088292 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" event={"ID":"5081902a-edad-4292-bdd8-99990a2fd4b6","Type":"ContainerStarted","Data":"b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31"} Jul 20 18:27:09.744926 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:09.744881 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:09.745916 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:09.745880 2567 patch_prober.go:28] interesting pod/e2e-derive-76c00924-openshift-default-fbc677687-clvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 20 18:27:09.746064 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:09.745942 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 20 18:27:10.745792 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:10.745755 2567 patch_prober.go:28] interesting pod/e2e-derive-76c00924-openshift-default-fbc677687-clvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 20 18:27:10.746822 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:10.746396 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 20 18:27:11.745862 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:11.745813 2567 patch_prober.go:28] interesting pod/e2e-derive-76c00924-openshift-default-fbc677687-clvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 20 18:27:11.746393 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:11.745885 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 20 18:27:12.745632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:12.745583 2567 patch_prober.go:28] interesting pod/e2e-derive-76c00924-openshift-default-fbc677687-clvkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 20 18:27:12.745858 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:12.745656 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 20 18:27:13.748748 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:13.748715 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:13.749254 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:13.749121 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:13.749763 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:13.749743 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:34.606024 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:34.605930 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf"] Jul 20 18:27:34.606599 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:34.606154 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" containerID="cri-o://b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31" gracePeriod=30 Jul 20 18:27:39.866199 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:39.866148 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:40.052458 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052427 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-certs\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052649 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052501 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-credential-socket\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052649 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052530 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-data\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052649 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052564 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft2ds\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-kube-api-access-ft2ds\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052649 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052590 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5081902a-edad-4292-bdd8-99990a2fd4b6-istiod-ca-cert\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052649 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052616 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-envoy\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052700 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-socket\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052730 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-token\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052751 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:27:40.052932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052766 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-podinfo\") pod \"5081902a-edad-4292-bdd8-99990a2fd4b6\" (UID: \"5081902a-edad-4292-bdd8-99990a2fd4b6\") " Jul 20 18:27:40.052932 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.052848 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:27:40.053213 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.053146 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:27:40.053213 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.053163 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-certs\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.053213 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.053210 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-credential-socket\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.053388 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.053211 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5081902a-edad-4292-bdd8-99990a2fd4b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:27:40.053388 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.053334 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:27:40.055115 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.055088 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:27:40.055443 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.055412 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-kube-api-access-ft2ds" (OuterVolumeSpecName: "kube-api-access-ft2ds") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "kube-api-access-ft2ds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:27:40.055443 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.055419 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 18:27:40.055567 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.055438 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "5081902a-edad-4292-bdd8-99990a2fd4b6" (UID: "5081902a-edad-4292-bdd8-99990a2fd4b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154270 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-data\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154317 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft2ds\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-kube-api-access-ft2ds\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154333 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5081902a-edad-4292-bdd8-99990a2fd4b6-istiod-ca-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154345 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-envoy\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154357 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5081902a-edad-4292-bdd8-99990a2fd4b6-workload-socket\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154368 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-token\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.154430 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.154395 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5081902a-edad-4292-bdd8-99990a2fd4b6-istio-podinfo\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:27:40.275453 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.275408 2567 generic.go:356] "Generic (PLEG): container finished" podID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerID="b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31" exitCode=0 Jul 20 18:27:40.275632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.275470 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" event={"ID":"5081902a-edad-4292-bdd8-99990a2fd4b6","Type":"ContainerDied","Data":"b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31"} Jul 20 18:27:40.275632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.275498 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" Jul 20 18:27:40.275632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.275522 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf" event={"ID":"5081902a-edad-4292-bdd8-99990a2fd4b6","Type":"ContainerDied","Data":"68a3e073814f121f658cb71991923804d17519e4695319f94e86d8b14246bbbf"} Jul 20 18:27:40.275632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.275551 2567 scope.go:117] "RemoveContainer" containerID="b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31" Jul 20 18:27:40.286693 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.286673 2567 scope.go:117] "RemoveContainer" containerID="f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213" Jul 20 18:27:40.295316 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.295297 2567 scope.go:117] "RemoveContainer" containerID="b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31" Jul 20 18:27:40.295580 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:27:40.295559 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31\": container with ID starting with b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31 not found: ID does not exist" containerID="b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31" Jul 20 18:27:40.295636 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.295592 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31"} err="failed to get container status \"b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31\": rpc error: code = NotFound desc = could not find container \"b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31\": container with ID starting with b1e3284776ca4b4c84454766e5ccbe1146e4462b088f6769c21edd5936d41d31 not found: ID does not exist" Jul 20 18:27:40.295636 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.295609 2567 scope.go:117] "RemoveContainer" containerID="f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213" Jul 20 18:27:40.295841 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:27:40.295819 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213\": container with ID starting with f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213 not found: ID does not exist" containerID="f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213" Jul 20 18:27:40.295881 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.295848 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213"} err="failed to get container status \"f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213\": rpc error: code = NotFound desc = could not find container \"f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213\": container with ID starting with f9cf38abc673bb093f13cf47736a3e26706f05bace79dcc14e8fb2dd1844d213 not found: ID does not exist" Jul 20 18:27:40.307282 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.307227 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf"] Jul 20 18:27:40.314839 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:40.314807 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-76c00924-openshift-default-fbc677687-clvkf"] Jul 20 18:27:41.441355 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:27:41.441313 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" path="/var/lib/kubelet/pods/5081902a-edad-4292-bdd8-99990a2fd4b6/volumes" Jul 20 18:29:29.310882 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.310849 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj"] Jul 20 18:29:29.311350 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.311334 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" Jul 20 18:29:29.311350 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.311351 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" Jul 20 18:29:29.311433 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.311427 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" Jul 20 18:29:29.311467 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.311436 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" Jul 20 18:29:29.314594 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.314576 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.317373 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.317352 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h2tfv\"" Jul 20 18:29:29.319595 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.319576 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e68a9f8e-serving-cert\"" Jul 20 18:29:29.329412 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.329382 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj"] Jul 20 18:29:29.419880 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.419845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg8jx\" (UniqueName: \"kubernetes.io/projected/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-kube-api-access-xg8jx\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.420075 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.419927 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-maas-api-tls\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.420075 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.419972 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-openshift-service-ca\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.521377 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.521333 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg8jx\" (UniqueName: \"kubernetes.io/projected/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-kube-api-access-xg8jx\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.521908 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.521884 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-maas-api-tls\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.521908 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.521937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-openshift-service-ca\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.522609 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.522588 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-openshift-service-ca\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.524469 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.524448 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-maas-api-tls\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.530678 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.530655 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg8jx\" (UniqueName: \"kubernetes.io/projected/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-kube-api-access-xg8jx\") pod \"maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.625022 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.624924 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:29.771322 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.771297 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj"] Jul 20 18:29:29.773537 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:29:29.773508 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce95716_dd7f_4f3c_8f40_6931ef10dc4d.slice/crio-ee05c8219b9a52e11dc3d0c55de8eb723de9cbc5b0f3517ebdd524bcfa3482ed WatchSource:0}: Error finding container ee05c8219b9a52e11dc3d0c55de8eb723de9cbc5b0f3517ebdd524bcfa3482ed: Status 404 returned error can't find the container with id ee05c8219b9a52e11dc3d0c55de8eb723de9cbc5b0f3517ebdd524bcfa3482ed Jul 20 18:29:29.937082 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:29.936993 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" event={"ID":"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d","Type":"ContainerStarted","Data":"ee05c8219b9a52e11dc3d0c55de8eb723de9cbc5b0f3517ebdd524bcfa3482ed"} Jul 20 18:29:30.945572 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:30.945538 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" event={"ID":"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d","Type":"ContainerStarted","Data":"d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a"} Jul 20 18:29:30.945990 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:30.945682 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:30.968679 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:30.968629 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" podStartSLOduration=1.461183557 podStartE2EDuration="1.96861304s" totalImagesPullingTime="507.429483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:29:30.967389355 +0000 UTC m=+2000.095506520" watchObservedRunningTime="2026-07-20 18:29:30.96861304 +0000 UTC m=+2000.096730206" Jul 20 18:29:36.957557 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:36.957523 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:51.619956 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:51.619921 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj"] Jul 20 18:29:51.620472 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:51.620129 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" podUID="2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" containerName="maas-api" containerID="cri-o://d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a" gracePeriod=30 Jul 20 18:29:51.886389 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:51.886329 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:52.039516 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.039478 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-openshift-service-ca\") pod \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " Jul 20 18:29:52.039516 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.039519 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-maas-api-tls\") pod \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " Jul 20 18:29:52.039735 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.039538 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg8jx\" (UniqueName: \"kubernetes.io/projected/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-kube-api-access-xg8jx\") pod \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\" (UID: \"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d\") " Jul 20 18:29:52.039924 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.039899 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" (UID: "2ce95716-dd7f-4f3c-8f40-6931ef10dc4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:29:52.041617 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.041593 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" (UID: "2ce95716-dd7f-4f3c-8f40-6931ef10dc4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:29:52.050371 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.041935 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-kube-api-access-xg8jx" (OuterVolumeSpecName: "kube-api-access-xg8jx") pod "2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" (UID: "2ce95716-dd7f-4f3c-8f40-6931ef10dc4d"). InnerVolumeSpecName "kube-api-access-xg8jx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:29:52.074783 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.074748 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" containerID="d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a" exitCode=0 Jul 20 18:29:52.074912 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.074804 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" Jul 20 18:29:52.074912 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.074833 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" event={"ID":"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d","Type":"ContainerDied","Data":"d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a"} Jul 20 18:29:52.074986 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.074917 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj" event={"ID":"2ce95716-dd7f-4f3c-8f40-6931ef10dc4d","Type":"ContainerDied","Data":"ee05c8219b9a52e11dc3d0c55de8eb723de9cbc5b0f3517ebdd524bcfa3482ed"} Jul 20 18:29:52.074986 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.074945 2567 scope.go:117] "RemoveContainer" containerID="d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a" Jul 20 18:29:52.084776 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.084756 2567 scope.go:117] "RemoveContainer" containerID="d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a" Jul 20 18:29:52.085035 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:29:52.085018 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a\": container with ID starting with d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a not found: ID does not exist" containerID="d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a" Jul 20 18:29:52.085088 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.085042 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a"} err="failed to get container status \"d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a\": rpc error: code = NotFound desc = could not find container \"d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a\": container with ID starting with d9b9e1179278f00212a8720637d88eddb513402030b2f9319237736ac0f5d78a not found: ID does not exist" Jul 20 18:29:52.097301 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.097270 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj"] Jul 20 18:29:52.101708 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.101684 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e68a9f8e-659d598884-tgmfj"] Jul 20 18:29:52.140485 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.140423 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-openshift-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:29:52.140485 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.140446 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-maas-api-tls\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:29:52.140485 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:52.140458 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg8jx\" (UniqueName: \"kubernetes.io/projected/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d-kube-api-access-xg8jx\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:29:53.441344 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:29:53.441309 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" path="/var/lib/kubelet/pods/2ce95716-dd7f-4f3c-8f40-6931ef10dc4d/volumes" Jul 20 18:31:11.525768 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:31:11.525742 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:31:11.536022 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:31:11.536002 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:33:35.176419 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.176324 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd"] Jul 20 18:33:35.177145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.176843 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" Jul 20 18:33:35.177145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.176861 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5081902a-edad-4292-bdd8-99990a2fd4b6" containerName="istio-proxy" Jul 20 18:33:35.177145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.176868 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" containerName="maas-api" Jul 20 18:33:35.177145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.176874 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" containerName="maas-api" Jul 20 18:33:35.177145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.176957 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce95716-dd7f-4f3c-8f40-6931ef10dc4d" containerName="maas-api" Jul 20 18:33:35.180383 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.180357 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.183400 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.183378 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h2tfv\"" Jul 20 18:33:35.183601 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.183583 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-02ddb5-serving-cert\"" Jul 20 18:33:35.191629 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.191604 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd"] Jul 20 18:33:35.259781 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.259747 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7r9\" (UniqueName: \"kubernetes.io/projected/97d9b4fe-50c1-4c72-8681-611cd3b46b41-kube-api-access-5l7r9\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.259971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.259800 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.259971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.259893 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97d9b4fe-50c1-4c72-8681-611cd3b46b41-openshift-service-ca\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.360882 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.360850 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97d9b4fe-50c1-4c72-8681-611cd3b46b41-openshift-service-ca\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.361083 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.360930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7r9\" (UniqueName: \"kubernetes.io/projected/97d9b4fe-50c1-4c72-8681-611cd3b46b41-kube-api-access-5l7r9\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.361083 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.361060 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.361230 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:33:35.361211 2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-serving-cert: secret "maas-api-e2e-inf-b-02ddb5-serving-cert" not found Jul 20 18:33:35.361310 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:33:35.361294 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls podName:97d9b4fe-50c1-4c72-8681-611cd3b46b41 nodeName:}" failed. No retries permitted until 2026-07-20 18:33:35.861276157 +0000 UTC m=+2244.989393301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls") pod "maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" (UID: "97d9b4fe-50c1-4c72-8681-611cd3b46b41") : secret "maas-api-e2e-inf-b-02ddb5-serving-cert" not found Jul 20 18:33:35.361601 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.361581 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97d9b4fe-50c1-4c72-8681-611cd3b46b41-openshift-service-ca\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.384509 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.384476 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7r9\" (UniqueName: \"kubernetes.io/projected/97d9b4fe-50c1-4c72-8681-611cd3b46b41-kube-api-access-5l7r9\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.867090 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.867054 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:35.869478 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:35.869453 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls\") pod \"maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:36.092452 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:36.092416 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:36.238069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:36.238041 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd"] Jul 20 18:33:36.240542 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:33:36.240517 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d9b4fe_50c1_4c72_8681_611cd3b46b41.slice/crio-6892c89949d44332ccb2791e8309b485f19034d8dcc2aabc5b5fb2e148d59fae WatchSource:0}: Error finding container 6892c89949d44332ccb2791e8309b485f19034d8dcc2aabc5b5fb2e148d59fae: Status 404 returned error can't find the container with id 6892c89949d44332ccb2791e8309b485f19034d8dcc2aabc5b5fb2e148d59fae Jul 20 18:33:36.241875 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:36.241857 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 18:33:36.406957 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:36.406864 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" event={"ID":"97d9b4fe-50c1-4c72-8681-611cd3b46b41","Type":"ContainerStarted","Data":"6892c89949d44332ccb2791e8309b485f19034d8dcc2aabc5b5fb2e148d59fae"} Jul 20 18:33:37.415066 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:37.415024 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" event={"ID":"97d9b4fe-50c1-4c72-8681-611cd3b46b41","Type":"ContainerStarted","Data":"000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21"} Jul 20 18:33:37.415507 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:37.415094 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:33:37.435091 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:37.435039 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" podStartSLOduration=2.093107651 podStartE2EDuration="2.435023095s" totalImagesPullingTime="341.915444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:33:37.431682533 +0000 UTC m=+2246.559799699" watchObservedRunningTime="2026-07-20 18:33:37.435023095 +0000 UTC m=+2246.563140261" Jul 20 18:33:43.426651 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:33:43.426619 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:35:11.548112 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.548021 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr"] Jul 20 18:35:11.551648 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.551630 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:11.554061 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.554040 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fk7rm\"" Jul 20 18:35:11.559152 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.559117 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr"] Jul 20 18:35:11.690095 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.690064 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeef9419-4db9-4453-b18f-3e1f3f71e4e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr\" (UID: \"aeef9419-4db9-4453-b18f-3e1f3f71e4e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:11.791232 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.791202 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeef9419-4db9-4453-b18f-3e1f3f71e4e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr\" (UID: \"aeef9419-4db9-4453-b18f-3e1f3f71e4e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:11.791873 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.791850 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeef9419-4db9-4453-b18f-3e1f3f71e4e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr\" (UID: \"aeef9419-4db9-4453-b18f-3e1f3f71e4e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:11.863691 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:11.863612 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:12.008655 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:12.008627 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr"] Jul 20 18:35:12.009848 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:35:12.009817 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeef9419_4db9_4453_b18f_3e1f3f71e4e9.slice/crio-51cac534a60cfab9f144db5beaccd38d2227a269b9be8adbc5095349d00f6511 WatchSource:0}: Error finding container 51cac534a60cfab9f144db5beaccd38d2227a269b9be8adbc5095349d00f6511: Status 404 returned error can't find the container with id 51cac534a60cfab9f144db5beaccd38d2227a269b9be8adbc5095349d00f6511 Jul 20 18:35:12.991581 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:12.991482 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" event={"ID":"aeef9419-4db9-4453-b18f-3e1f3f71e4e9","Type":"ContainerStarted","Data":"51cac534a60cfab9f144db5beaccd38d2227a269b9be8adbc5095349d00f6511"} Jul 20 18:35:15.007243 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:15.007135 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" event={"ID":"aeef9419-4db9-4453-b18f-3e1f3f71e4e9","Type":"ContainerStarted","Data":"c773ef69fab3eb4cf7daa996b36609336bd6aa88f490c024862fc999807accfc"} Jul 20 18:35:15.057538 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:15.057439 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" podStartSLOduration=1.638928061 podStartE2EDuration="4.057418621s" totalImagesPullingTime="2.41849056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:35:15.056745727 +0000 UTC m=+2344.184862893" watchObservedRunningTime="2026-07-20 18:35:15.057418621 +0000 UTC m=+2344.185535788" Jul 20 18:35:16.014079 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:16.014042 2567 generic.go:356] "Generic (PLEG): container finished" podID="aeef9419-4db9-4453-b18f-3e1f3f71e4e9" containerID="c773ef69fab3eb4cf7daa996b36609336bd6aa88f490c024862fc999807accfc" exitCode=0 Jul 20 18:35:16.014478 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:16.014125 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" event={"ID":"aeef9419-4db9-4453-b18f-3e1f3f71e4e9","Type":"ContainerDied","Data":"c773ef69fab3eb4cf7daa996b36609336bd6aa88f490c024862fc999807accfc"} Jul 20 18:35:17.153678 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:17.153652 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:17.247901 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:17.247872 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeef9419-4db9-4453-b18f-3e1f3f71e4e9-maas-api-service-ca\") pod \"aeef9419-4db9-4453-b18f-3e1f3f71e4e9\" (UID: \"aeef9419-4db9-4453-b18f-3e1f3f71e4e9\") " Jul 20 18:35:17.248406 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:17.248380 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeef9419-4db9-4453-b18f-3e1f3f71e4e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aeef9419-4db9-4453-b18f-3e1f3f71e4e9" (UID: "aeef9419-4db9-4453-b18f-3e1f3f71e4e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:35:17.349465 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:17.349429 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeef9419-4db9-4453-b18f-3e1f3f71e4e9-maas-api-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:35:18.027930 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:18.027894 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" event={"ID":"aeef9419-4db9-4453-b18f-3e1f3f71e4e9","Type":"ContainerDied","Data":"51cac534a60cfab9f144db5beaccd38d2227a269b9be8adbc5095349d00f6511"} Jul 20 18:35:18.027930 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:18.027926 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr" Jul 20 18:35:18.028147 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:18.027948 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51cac534a60cfab9f144db5beaccd38d2227a269b9be8adbc5095349d00f6511" Jul 20 18:35:26.654363 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:26.654325 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr"] Jul 20 18:35:26.657346 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:26.657314 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-adca0d-j68kr"] Jul 20 18:35:27.441781 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:27.441744 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeef9419-4db9-4453-b18f-3e1f3f71e4e9" path="/var/lib/kubelet/pods/aeef9419-4db9-4453-b18f-3e1f3f71e4e9/volumes" Jul 20 18:35:44.307668 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.307631 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd"] Jul 20 18:35:44.308546 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.308489 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" podUID="97d9b4fe-50c1-4c72-8681-611cd3b46b41" containerName="maas-api" containerID="cri-o://000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21" gracePeriod=30 Jul 20 18:35:44.767366 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.767341 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:35:44.818349 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.818318 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls\") pod \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " Jul 20 18:35:44.818530 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.818355 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l7r9\" (UniqueName: \"kubernetes.io/projected/97d9b4fe-50c1-4c72-8681-611cd3b46b41-kube-api-access-5l7r9\") pod \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " Jul 20 18:35:44.818530 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.818389 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97d9b4fe-50c1-4c72-8681-611cd3b46b41-openshift-service-ca\") pod \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\" (UID: \"97d9b4fe-50c1-4c72-8681-611cd3b46b41\") " Jul 20 18:35:44.818814 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.818788 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97d9b4fe-50c1-4c72-8681-611cd3b46b41-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97d9b4fe-50c1-4c72-8681-611cd3b46b41" (UID: "97d9b4fe-50c1-4c72-8681-611cd3b46b41"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:35:44.820463 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.820439 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d9b4fe-50c1-4c72-8681-611cd3b46b41-kube-api-access-5l7r9" (OuterVolumeSpecName: "kube-api-access-5l7r9") pod "97d9b4fe-50c1-4c72-8681-611cd3b46b41" (UID: "97d9b4fe-50c1-4c72-8681-611cd3b46b41"). InnerVolumeSpecName "kube-api-access-5l7r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:35:44.820657 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.820639 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97d9b4fe-50c1-4c72-8681-611cd3b46b41" (UID: "97d9b4fe-50c1-4c72-8681-611cd3b46b41"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 18:35:44.919868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.919788 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97d9b4fe-50c1-4c72-8681-611cd3b46b41-maas-api-tls\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:35:44.919868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.919816 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l7r9\" (UniqueName: \"kubernetes.io/projected/97d9b4fe-50c1-4c72-8681-611cd3b46b41-kube-api-access-5l7r9\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:35:44.919868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:44.919827 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97d9b4fe-50c1-4c72-8681-611cd3b46b41-openshift-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:35:45.191009 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.190922 2567 generic.go:356] "Generic (PLEG): container finished" podID="97d9b4fe-50c1-4c72-8681-611cd3b46b41" containerID="000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21" exitCode=0 Jul 20 18:35:45.191009 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.190971 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" Jul 20 18:35:45.191240 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.191007 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" event={"ID":"97d9b4fe-50c1-4c72-8681-611cd3b46b41","Type":"ContainerDied","Data":"000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21"} Jul 20 18:35:45.191240 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.191058 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd" event={"ID":"97d9b4fe-50c1-4c72-8681-611cd3b46b41","Type":"ContainerDied","Data":"6892c89949d44332ccb2791e8309b485f19034d8dcc2aabc5b5fb2e148d59fae"} Jul 20 18:35:45.191240 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.191079 2567 scope.go:117] "RemoveContainer" containerID="000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21" Jul 20 18:35:45.201739 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.201721 2567 scope.go:117] "RemoveContainer" containerID="000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21" Jul 20 18:35:45.202025 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:35:45.202007 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21\": container with ID starting with 000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21 not found: ID does not exist" containerID="000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21" Jul 20 18:35:45.202076 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.202034 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21"} err="failed to get container status \"000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21\": rpc error: code = NotFound desc = could not find container \"000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21\": container with ID starting with 000c53ed2af92fa9b632c8c4ace5d02fdec6ee1b191cb1f26c6ef49024d4ff21 not found: ID does not exist" Jul 20 18:35:45.227724 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.227697 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd"] Jul 20 18:35:45.245925 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.245898 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-02ddb5-76b5fd588-x6cmd"] Jul 20 18:35:45.440694 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:45.440657 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97d9b4fe-50c1-4c72-8681-611cd3b46b41" path="/var/lib/kubelet/pods/97d9b4fe-50c1-4c72-8681-611cd3b46b41/volumes" Jul 20 18:35:56.604027 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.603989 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn"] Jul 20 18:35:56.604509 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.604449 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d9b4fe-50c1-4c72-8681-611cd3b46b41" containerName="maas-api" Jul 20 18:35:56.604509 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.604461 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d9b4fe-50c1-4c72-8681-611cd3b46b41" containerName="maas-api" Jul 20 18:35:56.604509 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.604472 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeef9419-4db9-4453-b18f-3e1f3f71e4e9" containerName="revoke-keys" Jul 20 18:35:56.604509 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.604478 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeef9419-4db9-4453-b18f-3e1f3f71e4e9" containerName="revoke-keys" Jul 20 18:35:56.604641 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.604549 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d9b4fe-50c1-4c72-8681-611cd3b46b41" containerName="maas-api" Jul 20 18:35:56.604641 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.604558 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeef9419-4db9-4453-b18f-3e1f3f71e4e9" containerName="revoke-keys" Jul 20 18:35:56.611578 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.611555 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.614119 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.614100 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3054a0-openshift-default-dockercfg-69xpr\"" Jul 20 18:35:56.624757 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.624732 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn"] Jul 20 18:35:56.734837 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.734802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accc5164-77db-4e90-b6fb-7c7de924e1ba-istiod-ca-cert\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735026 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.734859 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-certs\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735026 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.734946 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-credential-socket\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735026 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.734989 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-token\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735026 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.735013 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-envoy\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.735054 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-data\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.735092 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-socket\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.735168 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-podinfo\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.735271 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.735224 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85r2z\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-kube-api-access-85r2z\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835707 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-credential-socket\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835707 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835714 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-token\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835732 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-envoy\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-data\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835774 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-socket\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835877 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-podinfo\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.835960 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85r2z\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-kube-api-access-85r2z\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836280 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.835966 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accc5164-77db-4e90-b6fb-7c7de924e1ba-istiod-ca-cert\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836280 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.836026 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-certs\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836280 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.836154 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-credential-socket\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836280 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.836271 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-socket\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836486 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.836311 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-data\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836486 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.836356 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-certs\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.836751 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.836731 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accc5164-77db-4e90-b6fb-7c7de924e1ba-istiod-ca-cert\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.838131 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.838110 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-envoy\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.838441 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.838419 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-podinfo\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.847755 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.847729 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-token\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.848365 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.848343 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85r2z\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-kube-api-access-85r2z\") pod \"e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:56.923436 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:56.923358 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:57.069948 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:35:57.069917 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaccc5164_77db_4e90_b6fb_7c7de924e1ba.slice/crio-4f8fa6f5d2091dbe0175d7fe588f0d6c164dcd5c28c6321536b6f640fb1c7784 WatchSource:0}: Error finding container 4f8fa6f5d2091dbe0175d7fe588f0d6c164dcd5c28c6321536b6f640fb1c7784: Status 404 returned error can't find the container with id 4f8fa6f5d2091dbe0175d7fe588f0d6c164dcd5c28c6321536b6f640fb1c7784 Jul 20 18:35:57.070728 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.070704 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn"] Jul 20 18:35:57.072102 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.072072 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:35:57.072152 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.072140 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:35:57.072232 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.072169 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:35:57.267606 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.267521 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" event={"ID":"accc5164-77db-4e90-b6fb-7c7de924e1ba","Type":"ContainerStarted","Data":"bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e"} Jul 20 18:35:57.267606 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.267593 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" event={"ID":"accc5164-77db-4e90-b6fb-7c7de924e1ba","Type":"ContainerStarted","Data":"4f8fa6f5d2091dbe0175d7fe588f0d6c164dcd5c28c6321536b6f640fb1c7784"} Jul 20 18:35:57.296899 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.296835 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podStartSLOduration=1.296815694 podStartE2EDuration="1.296815694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:35:57.294644221 +0000 UTC m=+2386.422761389" watchObservedRunningTime="2026-07-20 18:35:57.296815694 +0000 UTC m=+2386.424932861" Jul 20 18:35:57.924313 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:57.924273 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:58.929099 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:58.929068 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:59.279909 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:59.279819 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:35:59.281021 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:35:59.280995 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:11.368477 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.368445 2567 generic.go:356] "Generic (PLEG): container finished" podID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerID="bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e" exitCode=0 Jul 20 18:36:11.368916 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.368508 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" event={"ID":"accc5164-77db-4e90-b6fb-7c7de924e1ba","Type":"ContainerDied","Data":"bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e"} Jul 20 18:36:11.368965 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.368913 2567 scope.go:117] "RemoveContainer" containerID="bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e" Jul 20 18:36:11.370086 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.370054 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:36:11.370161 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.370143 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:36:11.370236 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.370204 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 18:36:11.569705 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.569679 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:36:11.582214 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:11.582162 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:36:12.377388 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:12.377355 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" event={"ID":"accc5164-77db-4e90-b6fb-7c7de924e1ba","Type":"ContainerStarted","Data":"474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77"} Jul 20 18:36:12.924360 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:12.924319 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:12.925269 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:12.925241 2567 patch_prober.go:28] interesting pod/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body= Jul 20 18:36:12.925349 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:12.925297 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" Jul 20 18:36:13.924622 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:13.924583 2567 patch_prober.go:28] interesting pod/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body= Jul 20 18:36:13.925003 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:13.924647 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" Jul 20 18:36:14.280861 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:14.280772 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:14.924555 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:14.924520 2567 patch_prober.go:28] interesting pod/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body= Jul 20 18:36:14.924754 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:14.924582 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" Jul 20 18:36:15.924145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:15.924111 2567 patch_prober.go:28] interesting pod/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body= Jul 20 18:36:15.924367 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:15.924166 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" Jul 20 18:36:16.927520 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:16.927485 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:16.928625 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:16.928606 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:46.355331 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:46.355246 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn"] Jul 20 18:36:46.355916 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:46.355461 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" containerID="cri-o://474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77" gracePeriod=30 Jul 20 18:36:46.930226 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:46.930158 2567 patch_prober.go:28] interesting pod/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 18:36:46.930394 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:46.930256 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 18:36:51.604952 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.604926 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:51.620604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.620535 2567 generic.go:356] "Generic (PLEG): container finished" podID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerID="474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77" exitCode=0 Jul 20 18:36:51.620604 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.620577 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" Jul 20 18:36:51.620748 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.620594 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" event={"ID":"accc5164-77db-4e90-b6fb-7c7de924e1ba","Type":"ContainerDied","Data":"474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77"} Jul 20 18:36:51.620748 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.620655 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn" event={"ID":"accc5164-77db-4e90-b6fb-7c7de924e1ba","Type":"ContainerDied","Data":"4f8fa6f5d2091dbe0175d7fe588f0d6c164dcd5c28c6321536b6f640fb1c7784"} Jul 20 18:36:51.620748 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.620681 2567 scope.go:117] "RemoveContainer" containerID="474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77" Jul 20 18:36:51.631335 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.631311 2567 scope.go:117] "RemoveContainer" containerID="bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e" Jul 20 18:36:51.641056 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.641022 2567 scope.go:117] "RemoveContainer" containerID="474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77" Jul 20 18:36:51.641356 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:36:51.641335 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77\": container with ID starting with 474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77 not found: ID does not exist" containerID="474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77" Jul 20 18:36:51.641438 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.641362 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77"} err="failed to get container status \"474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77\": rpc error: code = NotFound desc = could not find container \"474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77\": container with ID starting with 474cfe33dd398a9e8e83410cf25d8ed4b53263980ed41414543cf076d0c5ff77 not found: ID does not exist" Jul 20 18:36:51.641438 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.641379 2567 scope.go:117] "RemoveContainer" containerID="bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e" Jul 20 18:36:51.641645 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:36:51.641618 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e\": container with ID starting with bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e not found: ID does not exist" containerID="bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e" Jul 20 18:36:51.641701 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.641643 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e"} err="failed to get container status \"bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e\": rpc error: code = NotFound desc = could not find container \"bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e\": container with ID starting with bde5d381d3f80bed7b5c79344acc45b0d654518e9093ef54fcab3148f004356e not found: ID does not exist" Jul 20 18:36:51.783765 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783738 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-envoy\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.783937 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783776 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-token\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.783937 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783810 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85r2z\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-kube-api-access-85r2z\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.783937 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783852 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-data\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.783937 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783886 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-socket\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.783937 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783906 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accc5164-77db-4e90-b6fb-7c7de924e1ba-istiod-ca-cert\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.783937 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783926 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-certs\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.784325 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.783964 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-credential-socket\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.784325 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.784017 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-podinfo\") pod \"accc5164-77db-4e90-b6fb-7c7de924e1ba\" (UID: \"accc5164-77db-4e90-b6fb-7c7de924e1ba\") " Jul 20 18:36:51.784439 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.784392 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:36:51.784500 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.784423 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/accc5164-77db-4e90-b6fb-7c7de924e1ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:36:51.784648 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.784616 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:36:51.784779 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.784658 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:36:51.784875 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.784847 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:36:51.786109 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.786077 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 18:36:51.786109 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.786096 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-kube-api-access-85r2z" (OuterVolumeSpecName: "kube-api-access-85r2z") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "kube-api-access-85r2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:36:51.786305 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.786133 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:36:51.786305 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.786160 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "accc5164-77db-4e90-b6fb-7c7de924e1ba" (UID: "accc5164-77db-4e90-b6fb-7c7de924e1ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884896 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85r2z\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-kube-api-access-85r2z\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884921 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-data\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884930 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-socket\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884943 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/accc5164-77db-4e90-b6fb-7c7de924e1ba-istiod-ca-cert\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884957 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-workload-certs\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884965 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-credential-socket\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.884971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884973 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-podinfo\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.885297 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884981 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-envoy\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.885297 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.884988 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/accc5164-77db-4e90-b6fb-7c7de924e1ba-istio-token\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:36:51.945949 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.945919 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn"] Jul 20 18:36:51.950438 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:51.950416 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3054a0-openshift-default-76d5fdd845-47ssn"] Jul 20 18:36:53.441227 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:36:53.441173 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" path="/var/lib/kubelet/pods/accc5164-77db-4e90-b6fb-7c7de924e1ba/volumes" Jul 20 18:41:11.609979 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:41:11.609946 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:41:11.627347 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:41:11.627323 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:42:11.729515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:42:11.729480 2567 scope.go:117] "RemoveContainer" containerID="c773ef69fab3eb4cf7daa996b36609336bd6aa88f490c024862fc999807accfc" Jul 20 18:43:59.504803 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.504765 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140271145062192"] Jul 20 18:43:59.505296 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.505243 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" Jul 20 18:43:59.505296 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.505260 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" Jul 20 18:43:59.505296 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.505272 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" Jul 20 18:43:59.505296 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.505280 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" Jul 20 18:43:59.505447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.505349 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" Jul 20 18:43:59.505447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.505356 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="accc5164-77db-4e90-b6fb-7c7de924e1ba" containerName="istio-proxy" Jul 20 18:43:59.508379 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.508363 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:43:59.511200 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.511163 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lzmpz\"" Jul 20 18:43:59.515012 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.514985 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145062192"] Jul 20 18:43:59.589345 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.589314 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7slrs\" (UniqueName: \"kubernetes.io/projected/b27b5b56-be14-43fa-a6f6-39e8555c0e9d-kube-api-access-7slrs\") pod \"test-curl-7426-140271145062192\" (UID: \"b27b5b56-be14-43fa-a6f6-39e8555c0e9d\") " pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:43:59.690141 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.690100 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7slrs\" (UniqueName: \"kubernetes.io/projected/b27b5b56-be14-43fa-a6f6-39e8555c0e9d-kube-api-access-7slrs\") pod \"test-curl-7426-140271145062192\" (UID: \"b27b5b56-be14-43fa-a6f6-39e8555c0e9d\") " pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:43:59.698832 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.698805 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7slrs\" (UniqueName: \"kubernetes.io/projected/b27b5b56-be14-43fa-a6f6-39e8555c0e9d-kube-api-access-7slrs\") pod \"test-curl-7426-140271145062192\" (UID: \"b27b5b56-be14-43fa-a6f6-39e8555c0e9d\") " pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:43:59.819317 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.819284 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:43:59.960867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.960836 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145062192"] Jul 20 18:43:59.962252 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:43:59.962215 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb27b5b56_be14_43fa_a6f6_39e8555c0e9d.slice/crio-8665b27655c03eeddd00c2d5864603dd2e4e88c211964891f2c406089c9646e7 WatchSource:0}: Error finding container 8665b27655c03eeddd00c2d5864603dd2e4e88c211964891f2c406089c9646e7: Status 404 returned error can't find the container with id 8665b27655c03eeddd00c2d5864603dd2e4e88c211964891f2c406089c9646e7 Jul 20 18:43:59.963497 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:43:59.963479 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 18:44:00.186107 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:00.186018 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145062192" event={"ID":"b27b5b56-be14-43fa-a6f6-39e8555c0e9d","Type":"ContainerStarted","Data":"8665b27655c03eeddd00c2d5864603dd2e4e88c211964891f2c406089c9646e7"} Jul 20 18:44:01.193889 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:01.193855 2567 generic.go:356] "Generic (PLEG): container finished" podID="b27b5b56-be14-43fa-a6f6-39e8555c0e9d" containerID="a6edbfcd8bed7e92f71354016695a05d94ecfe18314cc2aa0da3e8c1e455c145" exitCode=0 Jul 20 18:44:01.194320 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:01.193921 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145062192" event={"ID":"b27b5b56-be14-43fa-a6f6-39e8555c0e9d","Type":"ContainerDied","Data":"a6edbfcd8bed7e92f71354016695a05d94ecfe18314cc2aa0da3e8c1e455c145"} Jul 20 18:44:02.343829 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.343804 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:44:02.383294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.383240 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140271145062192_b27b5b56-be14-43fa-a6f6-39e8555c0e9d/test-curl-7426-140271145062192/0.log" Jul 20 18:44:02.417346 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.417312 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7slrs\" (UniqueName: \"kubernetes.io/projected/b27b5b56-be14-43fa-a6f6-39e8555c0e9d-kube-api-access-7slrs\") pod \"b27b5b56-be14-43fa-a6f6-39e8555c0e9d\" (UID: \"b27b5b56-be14-43fa-a6f6-39e8555c0e9d\") " Jul 20 18:44:02.419564 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.419523 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27b5b56-be14-43fa-a6f6-39e8555c0e9d-kube-api-access-7slrs" (OuterVolumeSpecName: "kube-api-access-7slrs") pod "b27b5b56-be14-43fa-a6f6-39e8555c0e9d" (UID: "b27b5b56-be14-43fa-a6f6-39e8555c0e9d"). InnerVolumeSpecName "kube-api-access-7slrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:44:02.423659 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.423631 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140271145062192"] Jul 20 18:44:02.427871 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.427850 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140271145062192"] Jul 20 18:44:02.518398 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.518325 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7slrs\" (UniqueName: \"kubernetes.io/projected/b27b5b56-be14-43fa-a6f6-39e8555c0e9d-kube-api-access-7slrs\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:02.555886 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.555850 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140271145061552"] Jul 20 18:44:02.556336 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.556320 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27b5b56-be14-43fa-a6f6-39e8555c0e9d" containerName="test-curl-7426-140271145062192" Jul 20 18:44:02.556392 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.556337 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27b5b56-be14-43fa-a6f6-39e8555c0e9d" containerName="test-curl-7426-140271145062192" Jul 20 18:44:02.556438 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.556417 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27b5b56-be14-43fa-a6f6-39e8555c0e9d" containerName="test-curl-7426-140271145062192" Jul 20 18:44:02.559630 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.559605 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:02.566594 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.566564 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145061552"] Jul 20 18:44:02.619402 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.619371 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5md\" (UniqueName: \"kubernetes.io/projected/13a5b315-90ee-4403-aaef-1dcf94e5db4a-kube-api-access-lv5md\") pod \"test-curl-7426-140271145061552\" (UID: \"13a5b315-90ee-4403-aaef-1dcf94e5db4a\") " pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:02.720631 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.720591 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5md\" (UniqueName: \"kubernetes.io/projected/13a5b315-90ee-4403-aaef-1dcf94e5db4a-kube-api-access-lv5md\") pod \"test-curl-7426-140271145061552\" (UID: \"13a5b315-90ee-4403-aaef-1dcf94e5db4a\") " pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:02.729342 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.729316 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5md\" (UniqueName: \"kubernetes.io/projected/13a5b315-90ee-4403-aaef-1dcf94e5db4a-kube-api-access-lv5md\") pod \"test-curl-7426-140271145061552\" (UID: \"13a5b315-90ee-4403-aaef-1dcf94e5db4a\") " pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:02.870333 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:02.870303 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:03.009470 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:03.009444 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145061552"] Jul 20 18:44:03.011076 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:44:03.011045 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13a5b315_90ee_4403_aaef_1dcf94e5db4a.slice/crio-bca58ede5a67bc1a445203268aca6c4d0b00fe88f1b1dcb981c0fadefca8fe2d WatchSource:0}: Error finding container bca58ede5a67bc1a445203268aca6c4d0b00fe88f1b1dcb981c0fadefca8fe2d: Status 404 returned error can't find the container with id bca58ede5a67bc1a445203268aca6c4d0b00fe88f1b1dcb981c0fadefca8fe2d Jul 20 18:44:03.206471 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:03.206346 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:44:03.206471 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:03.206355 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8665b27655c03eeddd00c2d5864603dd2e4e88c211964891f2c406089c9646e7" Jul 20 18:44:03.207665 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:03.207637 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145061552" event={"ID":"13a5b315-90ee-4403-aaef-1dcf94e5db4a","Type":"ContainerStarted","Data":"bca58ede5a67bc1a445203268aca6c4d0b00fe88f1b1dcb981c0fadefca8fe2d"} Jul 20 18:44:03.218151 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:44:03.218124 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140271145062192\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="b27b5b56-be14-43fa-a6f6-39e8555c0e9d" pod="opendatahub/test-curl-7426-140271145062192" Jul 20 18:44:03.441613 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:03.441585 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b27b5b56-be14-43fa-a6f6-39e8555c0e9d" path="/var/lib/kubelet/pods/b27b5b56-be14-43fa-a6f6-39e8555c0e9d/volumes" Jul 20 18:44:04.215613 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:04.215575 2567 generic.go:356] "Generic (PLEG): container finished" podID="13a5b315-90ee-4403-aaef-1dcf94e5db4a" containerID="478eaef5c81410c74e40708ff99fad7aebc9a98ccf04b4636f14dbc7df7a1d75" exitCode=0 Jul 20 18:44:04.215837 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:04.215673 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145061552" event={"ID":"13a5b315-90ee-4403-aaef-1dcf94e5db4a","Type":"ContainerDied","Data":"478eaef5c81410c74e40708ff99fad7aebc9a98ccf04b4636f14dbc7df7a1d75"} Jul 20 18:44:05.363411 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.363383 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:05.389399 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.389363 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140271145061552_13a5b315-90ee-4403-aaef-1dcf94e5db4a/test-curl-7426-140271145061552/0.log" Jul 20 18:44:05.418649 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.418614 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140271145061552"] Jul 20 18:44:05.424364 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.424337 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140271145061552"] Jul 20 18:44:05.449331 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.449305 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv5md\" (UniqueName: \"kubernetes.io/projected/13a5b315-90ee-4403-aaef-1dcf94e5db4a-kube-api-access-lv5md\") pod \"13a5b315-90ee-4403-aaef-1dcf94e5db4a\" (UID: \"13a5b315-90ee-4403-aaef-1dcf94e5db4a\") " Jul 20 18:44:05.451424 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.451398 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13a5b315-90ee-4403-aaef-1dcf94e5db4a-kube-api-access-lv5md" (OuterVolumeSpecName: "kube-api-access-lv5md") pod "13a5b315-90ee-4403-aaef-1dcf94e5db4a" (UID: "13a5b315-90ee-4403-aaef-1dcf94e5db4a"). InnerVolumeSpecName "kube-api-access-lv5md". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:44:05.550227 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.550160 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv5md\" (UniqueName: \"kubernetes.io/projected/13a5b315-90ee-4403-aaef-1dcf94e5db4a-kube-api-access-lv5md\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:05.587921 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.587878 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:05.588396 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.588377 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13a5b315-90ee-4403-aaef-1dcf94e5db4a" containerName="test-curl-7426-140271145061552" Jul 20 18:44:05.588501 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.588399 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="13a5b315-90ee-4403-aaef-1dcf94e5db4a" containerName="test-curl-7426-140271145061552" Jul 20 18:44:05.588568 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.588526 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="13a5b315-90ee-4403-aaef-1dcf94e5db4a" containerName="test-curl-7426-140271145061552" Jul 20 18:44:05.592124 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.592102 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:05.598697 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.598667 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:05.651289 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.651250 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f24fc\" (UniqueName: \"kubernetes.io/projected/fefbb056-f8af-4abe-978a-003ce3684e4f-kube-api-access-f24fc\") pod \"test-curl-7426-140271145048512\" (UID: \"fefbb056-f8af-4abe-978a-003ce3684e4f\") " pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:05.752912 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.752876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f24fc\" (UniqueName: \"kubernetes.io/projected/fefbb056-f8af-4abe-978a-003ce3684e4f-kube-api-access-f24fc\") pod \"test-curl-7426-140271145048512\" (UID: \"fefbb056-f8af-4abe-978a-003ce3684e4f\") " pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:05.768949 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.768915 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f24fc\" (UniqueName: \"kubernetes.io/projected/fefbb056-f8af-4abe-978a-003ce3684e4f-kube-api-access-f24fc\") pod \"test-curl-7426-140271145048512\" (UID: \"fefbb056-f8af-4abe-978a-003ce3684e4f\") " pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:05.902965 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:05.902872 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:06.062437 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:06.062404 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:06.065433 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:44:06.065391 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfefbb056_f8af_4abe_978a_003ce3684e4f.slice/crio-bc337a39711ff64e55dae4c623444ccbd64de8312b6c1e5ac2ed1c9ce8559cac WatchSource:0}: Error finding container bc337a39711ff64e55dae4c623444ccbd64de8312b6c1e5ac2ed1c9ce8559cac: Status 404 returned error can't find the container with id bc337a39711ff64e55dae4c623444ccbd64de8312b6c1e5ac2ed1c9ce8559cac Jul 20 18:44:06.228069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:06.227983 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145061552" Jul 20 18:44:06.228069 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:06.228031 2567 scope.go:117] "RemoveContainer" containerID="478eaef5c81410c74e40708ff99fad7aebc9a98ccf04b4636f14dbc7df7a1d75" Jul 20 18:44:06.229417 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:06.229336 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145048512" event={"ID":"fefbb056-f8af-4abe-978a-003ce3684e4f","Type":"ContainerStarted","Data":"bc337a39711ff64e55dae4c623444ccbd64de8312b6c1e5ac2ed1c9ce8559cac"} Jul 20 18:44:07.238230 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:07.238172 2567 generic.go:356] "Generic (PLEG): container finished" podID="fefbb056-f8af-4abe-978a-003ce3684e4f" containerID="5880c0ec936d17f8894f83754ac22a7369832f09a63c2d03efec20e3225d53fa" exitCode=0 Jul 20 18:44:07.238613 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:07.238225 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145048512" event={"ID":"fefbb056-f8af-4abe-978a-003ce3684e4f","Type":"ContainerDied","Data":"5880c0ec936d17f8894f83754ac22a7369832f09a63c2d03efec20e3225d53fa"} Jul 20 18:44:07.441151 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:07.441119 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13a5b315-90ee-4403-aaef-1dcf94e5db4a" path="/var/lib/kubelet/pods/13a5b315-90ee-4403-aaef-1dcf94e5db4a/volumes" Jul 20 18:44:08.387386 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.387360 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:08.412226 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.412195 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140271145048512_fefbb056-f8af-4abe-978a-003ce3684e4f/test-curl-7426-140271145048512/0.log" Jul 20 18:44:08.438696 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.438661 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:08.444927 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.444895 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:08.484482 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.484452 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f24fc\" (UniqueName: \"kubernetes.io/projected/fefbb056-f8af-4abe-978a-003ce3684e4f-kube-api-access-f24fc\") pod \"fefbb056-f8af-4abe-978a-003ce3684e4f\" (UID: \"fefbb056-f8af-4abe-978a-003ce3684e4f\") " Jul 20 18:44:08.486590 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.486568 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fefbb056-f8af-4abe-978a-003ce3684e4f-kube-api-access-f24fc" (OuterVolumeSpecName: "kube-api-access-f24fc") pod "fefbb056-f8af-4abe-978a-003ce3684e4f" (UID: "fefbb056-f8af-4abe-978a-003ce3684e4f"). InnerVolumeSpecName "kube-api-access-f24fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:44:08.566892 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.566860 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:08.567323 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.567309 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fefbb056-f8af-4abe-978a-003ce3684e4f" containerName="test-curl-7426-140271145048512" Jul 20 18:44:08.567389 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.567325 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fefbb056-f8af-4abe-978a-003ce3684e4f" containerName="test-curl-7426-140271145048512" Jul 20 18:44:08.567425 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.567408 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fefbb056-f8af-4abe-978a-003ce3684e4f" containerName="test-curl-7426-140271145048512" Jul 20 18:44:08.570473 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.570457 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:08.578775 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.578747 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:08.586297 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.585961 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f24fc\" (UniqueName: \"kubernetes.io/projected/fefbb056-f8af-4abe-978a-003ce3684e4f-kube-api-access-f24fc\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:08.687513 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.687470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc86g\" (UniqueName: \"kubernetes.io/projected/90b8980b-a7f8-40cd-8d3d-c338f41b831c-kube-api-access-rc86g\") pod \"test-curl-7426-140271145048512\" (UID: \"90b8980b-a7f8-40cd-8d3d-c338f41b831c\") " pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:08.789032 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.788994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc86g\" (UniqueName: \"kubernetes.io/projected/90b8980b-a7f8-40cd-8d3d-c338f41b831c-kube-api-access-rc86g\") pod \"test-curl-7426-140271145048512\" (UID: \"90b8980b-a7f8-40cd-8d3d-c338f41b831c\") " pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:08.798320 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.798293 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc86g\" (UniqueName: \"kubernetes.io/projected/90b8980b-a7f8-40cd-8d3d-c338f41b831c-kube-api-access-rc86g\") pod \"test-curl-7426-140271145048512\" (UID: \"90b8980b-a7f8-40cd-8d3d-c338f41b831c\") " pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:08.880049 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:08.879954 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:09.041792 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:09.041764 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:09.043165 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:44:09.043141 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90b8980b_a7f8_40cd_8d3d_c338f41b831c.slice/crio-c14267cc4f4d087201cb9fd0353494bdcd93070d1b001572cfae2359894fc523 WatchSource:0}: Error finding container c14267cc4f4d087201cb9fd0353494bdcd93070d1b001572cfae2359894fc523: Status 404 returned error can't find the container with id c14267cc4f4d087201cb9fd0353494bdcd93070d1b001572cfae2359894fc523 Jul 20 18:44:09.251382 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:09.251297 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:09.251382 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:09.251336 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc337a39711ff64e55dae4c623444ccbd64de8312b6c1e5ac2ed1c9ce8559cac" Jul 20 18:44:09.252503 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:09.252476 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145048512" event={"ID":"90b8980b-a7f8-40cd-8d3d-c338f41b831c","Type":"ContainerStarted","Data":"c14267cc4f4d087201cb9fd0353494bdcd93070d1b001572cfae2359894fc523"} Jul 20 18:44:09.264694 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:09.264665 2567 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7426-140271145048512" oldPodUID="fefbb056-f8af-4abe-978a-003ce3684e4f" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" Jul 20 18:44:09.442129 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:09.442100 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fefbb056-f8af-4abe-978a-003ce3684e4f" path="/var/lib/kubelet/pods/fefbb056-f8af-4abe-978a-003ce3684e4f/volumes" Jul 20 18:44:10.260384 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:10.260344 2567 generic.go:356] "Generic (PLEG): container finished" podID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" containerID="c24f9fc5b5dc657dd6fd2c4e49bb7b2ef9e3bde53e91f0543cea957ce56cdcd1" exitCode=0 Jul 20 18:44:10.260587 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:10.260426 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271145048512" event={"ID":"90b8980b-a7f8-40cd-8d3d-c338f41b831c","Type":"ContainerDied","Data":"c24f9fc5b5dc657dd6fd2c4e49bb7b2ef9e3bde53e91f0543cea957ce56cdcd1"} Jul 20 18:44:11.408370 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.408343 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:11.434018 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.433989 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140271145048512_90b8980b-a7f8-40cd-8d3d-c338f41b831c/test-curl-7426-140271145048512/0.log" Jul 20 18:44:11.461310 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.460256 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:11.470278 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.470243 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140271145048512"] Jul 20 18:44:11.516380 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.516345 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc86g\" (UniqueName: \"kubernetes.io/projected/90b8980b-a7f8-40cd-8d3d-c338f41b831c-kube-api-access-rc86g\") pod \"90b8980b-a7f8-40cd-8d3d-c338f41b831c\" (UID: \"90b8980b-a7f8-40cd-8d3d-c338f41b831c\") " Jul 20 18:44:11.518489 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.518434 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b8980b-a7f8-40cd-8d3d-c338f41b831c-kube-api-access-rc86g" (OuterVolumeSpecName: "kube-api-access-rc86g") pod "90b8980b-a7f8-40cd-8d3d-c338f41b831c" (UID: "90b8980b-a7f8-40cd-8d3d-c338f41b831c"). InnerVolumeSpecName "kube-api-access-rc86g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:44:11.596157 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.596122 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:11.596587 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.596572 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" containerName="test-curl-7426-140271145048512" Jul 20 18:44:11.596637 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.596589 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" containerName="test-curl-7426-140271145048512" Jul 20 18:44:11.596681 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.596671 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" containerName="test-curl-7426-140271145048512" Jul 20 18:44:11.599681 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.599664 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:11.604443 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.604418 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:11.617545 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.617514 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc86g\" (UniqueName: \"kubernetes.io/projected/90b8980b-a7f8-40cd-8d3d-c338f41b831c-kube-api-access-rc86g\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:11.718501 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.718464 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbxb\" (UniqueName: \"kubernetes.io/projected/e53e62b0-6c88-45e3-a883-939a34e76ebb-kube-api-access-8rbxb\") pod \"test-curl-7426-140271146595360\" (UID: \"e53e62b0-6c88-45e3-a883-939a34e76ebb\") " pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:11.819238 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.819164 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbxb\" (UniqueName: \"kubernetes.io/projected/e53e62b0-6c88-45e3-a883-939a34e76ebb-kube-api-access-8rbxb\") pod \"test-curl-7426-140271146595360\" (UID: \"e53e62b0-6c88-45e3-a883-939a34e76ebb\") " pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:11.827599 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.827574 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbxb\" (UniqueName: \"kubernetes.io/projected/e53e62b0-6c88-45e3-a883-939a34e76ebb-kube-api-access-8rbxb\") pod \"test-curl-7426-140271146595360\" (UID: \"e53e62b0-6c88-45e3-a883-939a34e76ebb\") " pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:11.910807 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:11.910766 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:12.057100 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:12.057065 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:12.058236 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:44:12.058211 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode53e62b0_6c88_45e3_a883_939a34e76ebb.slice/crio-c822d0274e5bdbc6bc63e0027d9fe9464639dab3d09b73ac5ca2a1c3376f9b56 WatchSource:0}: Error finding container c822d0274e5bdbc6bc63e0027d9fe9464639dab3d09b73ac5ca2a1c3376f9b56: Status 404 returned error can't find the container with id c822d0274e5bdbc6bc63e0027d9fe9464639dab3d09b73ac5ca2a1c3376f9b56 Jul 20 18:44:12.277154 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:12.277112 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271146595360" event={"ID":"e53e62b0-6c88-45e3-a883-939a34e76ebb","Type":"ContainerStarted","Data":"c822d0274e5bdbc6bc63e0027d9fe9464639dab3d09b73ac5ca2a1c3376f9b56"} Jul 20 18:44:12.278412 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:12.278390 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:12.278538 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:12.278419 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c14267cc4f4d087201cb9fd0353494bdcd93070d1b001572cfae2359894fc523" Jul 20 18:44:12.289654 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:44:12.289624 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140271145048512\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:13.286632 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:13.286597 2567 generic.go:356] "Generic (PLEG): container finished" podID="e53e62b0-6c88-45e3-a883-939a34e76ebb" containerID="2e2a8194bc79133e2b62cdfaa6e9170a09b10ae77c7b301930c193e317f96483" exitCode=0 Jul 20 18:44:13.287084 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:13.286632 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271146595360" event={"ID":"e53e62b0-6c88-45e3-a883-939a34e76ebb","Type":"ContainerDied","Data":"2e2a8194bc79133e2b62cdfaa6e9170a09b10ae77c7b301930c193e317f96483"} Jul 20 18:44:13.306859 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:44:13.306816 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140271145048512\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" pod="opendatahub/test-curl-7426-140271145048512" Jul 20 18:44:13.441784 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:13.441744 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b8980b-a7f8-40cd-8d3d-c338f41b831c" path="/var/lib/kubelet/pods/90b8980b-a7f8-40cd-8d3d-c338f41b831c/volumes" Jul 20 18:44:14.438432 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.438406 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:14.461140 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.461106 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140271146595360_e53e62b0-6c88-45e3-a883-939a34e76ebb/test-curl-7426-140271146595360/0.log" Jul 20 18:44:14.487436 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.487400 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:14.492992 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.492964 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:14.543354 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.543278 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rbxb\" (UniqueName: \"kubernetes.io/projected/e53e62b0-6c88-45e3-a883-939a34e76ebb-kube-api-access-8rbxb\") pod \"e53e62b0-6c88-45e3-a883-939a34e76ebb\" (UID: \"e53e62b0-6c88-45e3-a883-939a34e76ebb\") " Jul 20 18:44:14.545380 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.545355 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53e62b0-6c88-45e3-a883-939a34e76ebb-kube-api-access-8rbxb" (OuterVolumeSpecName: "kube-api-access-8rbxb") pod "e53e62b0-6c88-45e3-a883-939a34e76ebb" (UID: "e53e62b0-6c88-45e3-a883-939a34e76ebb"). InnerVolumeSpecName "kube-api-access-8rbxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:44:14.617168 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.617125 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:14.617722 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.617701 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e53e62b0-6c88-45e3-a883-939a34e76ebb" containerName="test-curl-7426-140271146595360" Jul 20 18:44:14.617832 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.617723 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e53e62b0-6c88-45e3-a883-939a34e76ebb" containerName="test-curl-7426-140271146595360" Jul 20 18:44:14.617885 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.617835 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e53e62b0-6c88-45e3-a883-939a34e76ebb" containerName="test-curl-7426-140271146595360" Jul 20 18:44:14.620947 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.620925 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:14.627790 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.627764 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:14.645004 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.644977 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rbxb\" (UniqueName: \"kubernetes.io/projected/e53e62b0-6c88-45e3-a883-939a34e76ebb-kube-api-access-8rbxb\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:14.745549 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.745511 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd69v\" (UniqueName: \"kubernetes.io/projected/c07aeac4-03fd-4192-b868-51645e77199c-kube-api-access-qd69v\") pod \"test-curl-7426-140271146595360\" (UID: \"c07aeac4-03fd-4192-b868-51645e77199c\") " pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:14.847137 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.847105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd69v\" (UniqueName: \"kubernetes.io/projected/c07aeac4-03fd-4192-b868-51645e77199c-kube-api-access-qd69v\") pod \"test-curl-7426-140271146595360\" (UID: \"c07aeac4-03fd-4192-b868-51645e77199c\") " pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:14.857389 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.857359 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd69v\" (UniqueName: \"kubernetes.io/projected/c07aeac4-03fd-4192-b868-51645e77199c-kube-api-access-qd69v\") pod \"test-curl-7426-140271146595360\" (UID: \"c07aeac4-03fd-4192-b868-51645e77199c\") " pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:14.931427 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:14.931395 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:15.072037 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:15.072006 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:15.073092 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:44:15.073072 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc07aeac4_03fd_4192_b868_51645e77199c.slice/crio-e620f10434e8f6c5bc1cbb77378e383f6eb43a3df86c85ddbb823ba4b0711d02 WatchSource:0}: Error finding container e620f10434e8f6c5bc1cbb77378e383f6eb43a3df86c85ddbb823ba4b0711d02: Status 404 returned error can't find the container with id e620f10434e8f6c5bc1cbb77378e383f6eb43a3df86c85ddbb823ba4b0711d02 Jul 20 18:44:15.299629 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:15.299595 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271146595360" event={"ID":"c07aeac4-03fd-4192-b868-51645e77199c","Type":"ContainerStarted","Data":"e620f10434e8f6c5bc1cbb77378e383f6eb43a3df86c85ddbb823ba4b0711d02"} Jul 20 18:44:15.300940 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:15.300920 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:15.300940 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:15.300936 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c822d0274e5bdbc6bc63e0027d9fe9464639dab3d09b73ac5ca2a1c3376f9b56" Jul 20 18:44:15.321973 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:15.321937 2567 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7426-140271146595360" oldPodUID="e53e62b0-6c88-45e3-a883-939a34e76ebb" podUID="c07aeac4-03fd-4192-b868-51645e77199c" Jul 20 18:44:15.442262 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:15.442226 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e53e62b0-6c88-45e3-a883-939a34e76ebb" path="/var/lib/kubelet/pods/e53e62b0-6c88-45e3-a883-939a34e76ebb/volumes" Jul 20 18:44:16.308268 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:16.308232 2567 generic.go:356] "Generic (PLEG): container finished" podID="c07aeac4-03fd-4192-b868-51645e77199c" containerID="abbce32a527a9b68b6016e236536e3e5f224c4c4911743fdb2b1b73f9101001c" exitCode=0 Jul 20 18:44:16.308475 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:16.308276 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140271146595360" event={"ID":"c07aeac4-03fd-4192-b868-51645e77199c","Type":"ContainerDied","Data":"abbce32a527a9b68b6016e236536e3e5f224c4c4911743fdb2b1b73f9101001c"} Jul 20 18:44:17.461225 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.461197 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:17.483784 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.483728 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140271146595360_c07aeac4-03fd-4192-b868-51645e77199c/test-curl-7426-140271146595360/0.log" Jul 20 18:44:17.511166 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.511131 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:17.513609 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.513589 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140271146595360"] Jul 20 18:44:17.583355 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.583253 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd69v\" (UniqueName: \"kubernetes.io/projected/c07aeac4-03fd-4192-b868-51645e77199c-kube-api-access-qd69v\") pod \"c07aeac4-03fd-4192-b868-51645e77199c\" (UID: \"c07aeac4-03fd-4192-b868-51645e77199c\") " Jul 20 18:44:17.585318 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.585289 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07aeac4-03fd-4192-b868-51645e77199c-kube-api-access-qd69v" (OuterVolumeSpecName: "kube-api-access-qd69v") pod "c07aeac4-03fd-4192-b868-51645e77199c" (UID: "c07aeac4-03fd-4192-b868-51645e77199c"). InnerVolumeSpecName "kube-api-access-qd69v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 18:44:17.684815 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.684765 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd69v\" (UniqueName: \"kubernetes.io/projected/c07aeac4-03fd-4192-b868-51645e77199c-kube-api-access-qd69v\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:17.695426 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.695389 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p"] Jul 20 18:44:17.695835 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.695821 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c07aeac4-03fd-4192-b868-51645e77199c" containerName="test-curl-7426-140271146595360" Jul 20 18:44:17.695835 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.695835 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c07aeac4-03fd-4192-b868-51645e77199c" containerName="test-curl-7426-140271146595360" Jul 20 18:44:17.695971 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.695910 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c07aeac4-03fd-4192-b868-51645e77199c" containerName="test-curl-7426-140271146595360" Jul 20 18:44:17.699228 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.699211 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:17.701903 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.701882 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fk7rm\"" Jul 20 18:44:17.706413 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.706380 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p"] Jul 20 18:44:17.785860 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.785820 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/849ac704-cac1-4fa0-93e4-63a50c3c7ca3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p\" (UID: \"849ac704-cac1-4fa0-93e4-63a50c3c7ca3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:17.887455 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.887366 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/849ac704-cac1-4fa0-93e4-63a50c3c7ca3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p\" (UID: \"849ac704-cac1-4fa0-93e4-63a50c3c7ca3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:17.888013 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:17.887993 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/849ac704-cac1-4fa0-93e4-63a50c3c7ca3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p\" (UID: \"849ac704-cac1-4fa0-93e4-63a50c3c7ca3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:18.010539 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:18.010501 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:18.150228 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:18.150172 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p"] Jul 20 18:44:18.151544 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:44:18.151511 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849ac704_cac1_4fa0_93e4_63a50c3c7ca3.slice/crio-555f6f39fe453903cfab1275f51021fa6b5868d470c64cfe78e143e86a742917 WatchSource:0}: Error finding container 555f6f39fe453903cfab1275f51021fa6b5868d470c64cfe78e143e86a742917: Status 404 returned error can't find the container with id 555f6f39fe453903cfab1275f51021fa6b5868d470c64cfe78e143e86a742917 Jul 20 18:44:18.327071 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:18.327046 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:18.327257 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:18.327093 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e620f10434e8f6c5bc1cbb77378e383f6eb43a3df86c85ddbb823ba4b0711d02" Jul 20 18:44:18.328444 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:18.328419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" event={"ID":"849ac704-cac1-4fa0-93e4-63a50c3c7ca3","Type":"ContainerStarted","Data":"555f6f39fe453903cfab1275f51021fa6b5868d470c64cfe78e143e86a742917"} Jul 20 18:44:18.337827 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:44:18.337800 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140271146595360\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="c07aeac4-03fd-4192-b868-51645e77199c" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:19.335448 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:19.335415 2567 generic.go:356] "Generic (PLEG): container finished" podID="849ac704-cac1-4fa0-93e4-63a50c3c7ca3" containerID="ee51de8d18815ab41054ddcbabb4249af219ef01942b212e4833d6ea7d6527bb" exitCode=0 Jul 20 18:44:19.335830 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:19.335498 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" event={"ID":"849ac704-cac1-4fa0-93e4-63a50c3c7ca3","Type":"ContainerDied","Data":"ee51de8d18815ab41054ddcbabb4249af219ef01942b212e4833d6ea7d6527bb"} Jul 20 18:44:19.338048 ip-10-0-133-135 kubenswrapper[2567]: E0720 18:44:19.338022 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140271146595360\" is forbidden: User \"system:node:ip-10-0-133-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-135.ec2.internal' and this object" podUID="c07aeac4-03fd-4192-b868-51645e77199c" pod="opendatahub/test-curl-7426-140271146595360" Jul 20 18:44:19.441868 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:19.441834 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c07aeac4-03fd-4192-b868-51645e77199c" path="/var/lib/kubelet/pods/c07aeac4-03fd-4192-b868-51645e77199c/volumes" Jul 20 18:44:20.481039 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:20.481009 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:20.613562 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:20.612751 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/849ac704-cac1-4fa0-93e4-63a50c3c7ca3-maas-api-service-ca\") pod \"849ac704-cac1-4fa0-93e4-63a50c3c7ca3\" (UID: \"849ac704-cac1-4fa0-93e4-63a50c3c7ca3\") " Jul 20 18:44:20.614208 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:20.613897 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849ac704-cac1-4fa0-93e4-63a50c3c7ca3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "849ac704-cac1-4fa0-93e4-63a50c3c7ca3" (UID: "849ac704-cac1-4fa0-93e4-63a50c3c7ca3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 18:44:20.714228 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:20.714169 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/849ac704-cac1-4fa0-93e4-63a50c3c7ca3-maas-api-service-ca\") on node \"ip-10-0-133-135.ec2.internal\" DevicePath \"\"" Jul 20 18:44:21.349089 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:21.349059 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" Jul 20 18:44:21.349276 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:21.349083 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p" event={"ID":"849ac704-cac1-4fa0-93e4-63a50c3c7ca3","Type":"ContainerDied","Data":"555f6f39fe453903cfab1275f51021fa6b5868d470c64cfe78e143e86a742917"} Jul 20 18:44:21.349276 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:21.349157 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="555f6f39fe453903cfab1275f51021fa6b5868d470c64cfe78e143e86a742917" Jul 20 18:44:32.821139 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:32.821095 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p"] Jul 20 18:44:32.823747 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:32.823721 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0d90bc-fwc6p"] Jul 20 18:44:33.441353 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:44:33.441320 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849ac704-cac1-4fa0-93e4-63a50c3c7ca3" path="/var/lib/kubelet/pods/849ac704-cac1-4fa0-93e4-63a50c3c7ca3/volumes" Jul 20 18:45:03.715980 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:03.715947 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9f5557f5d-28kvz_f4415671-9428-4b45-8ba9-5092769a1ccb/authorino/0.log" Jul 20 18:45:10.018059 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:10.018028 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gx8bj_5d4341e7-78f1-442b-aabb-6b381486e766/manager/0.log" Jul 20 18:45:10.148892 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:10.148862 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2gdzv_0c757ec3-9d16-4264-97d2-4758e47d8703/manager/0.log" Jul 20 18:45:10.426715 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:10.426686 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mwntz_dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e/server/0.log" Jul 20 18:45:10.561864 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:10.561834 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-drdk6_1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4/manager/0.log" Jul 20 18:45:10.835503 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:10.835477 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79747b4d4c-dqbms_e190c526-deae-442b-bb20-7edbafa87611/copy-manifests/0.log" Jul 20 18:45:10.871820 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:10.871789 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79747b4d4c-dqbms_e190c526-deae-442b-bb20-7edbafa87611/manager/0.log" Jul 20 18:45:11.939115 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:11.939085 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wb78n_1533c7d0-ebd6-453a-8465-4263a389e422/postgres/0.log" Jul 20 18:45:12.774198 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:12.774160 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg_0558c5bb-18b8-4534-9087-e203a0276770/util/0.log" Jul 20 18:45:12.786419 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:12.786388 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg_0558c5bb-18b8-4534-9087-e203a0276770/pull/0.log" Jul 20 18:45:12.793915 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:12.793887 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg_0558c5bb-18b8-4534-9087-e203a0276770/extract/0.log" Jul 20 18:45:12.908323 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:12.908297 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx_9decc64d-d9d2-41cc-a07f-3b407a61738f/util/0.log" Jul 20 18:45:12.915771 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:12.915744 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx_9decc64d-d9d2-41cc-a07f-3b407a61738f/pull/0.log" Jul 20 18:45:12.923191 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:12.923155 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx_9decc64d-d9d2-41cc-a07f-3b407a61738f/extract/0.log" Jul 20 18:45:13.046294 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.046209 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw_4a45a589-a43d-4b2a-8eb6-b69c460e6f6e/util/0.log" Jul 20 18:45:13.054393 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.054359 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw_4a45a589-a43d-4b2a-8eb6-b69c460e6f6e/pull/0.log" Jul 20 18:45:13.063232 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.063207 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw_4a45a589-a43d-4b2a-8eb6-b69c460e6f6e/extract/0.log" Jul 20 18:45:13.202861 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.202835 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z_d55eaab4-37b9-4792-9a21-8c3e77bf5710/extract/0.log" Jul 20 18:45:13.214978 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.214956 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z_d55eaab4-37b9-4792-9a21-8c3e77bf5710/util/0.log" Jul 20 18:45:13.231296 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.231268 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z_d55eaab4-37b9-4792-9a21-8c3e77bf5710/pull/0.log" Jul 20 18:45:13.368546 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.368420 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9f5557f5d-28kvz_f4415671-9428-4b45-8ba9-5092769a1ccb/authorino/0.log" Jul 20 18:45:13.525896 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.525864 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kmq6z_b2e119da-1f94-4777-9107-2f152558ab06/manager/0.log" Jul 20 18:45:13.658318 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.658238 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lq9vh_945777a1-35fb-4ae0-83f9-90584f9adaf7/manager/0.log" Jul 20 18:45:13.916303 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:13.916222 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lfw48_1ccf85ab-fb69-4e69-ba1c-65d079d9086b/registry-server/0.log" Jul 20 18:45:14.362267 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:14.362239 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4j4jh_99a3c160-632a-4a1b-8f76-c20917e9c112/manager/0.log" Jul 20 18:45:14.747845 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:14.747748 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4d7j47_57aadb03-24ae-414a-80d7-6aa445139129/istio-proxy/0.log" Jul 20 18:45:17.157634 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:17.157593 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75cc6b94bb-szq28_e5d8d836-6592-4308-ac78-e6bd720f66fd/kube-auth-proxy/0.log" Jul 20 18:45:17.437759 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:17.437676 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/1.log" Jul 20 18:45:17.563498 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:17.563469 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wp2ks_9b9575a6-2fca-43e5-b804-1e5e32f9dc68/payload-pre-processing/0.log" Jul 20 18:45:18.241031 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:18.241000 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5nm5f_94b2cfc1-12b2-4d23-b444-63ccc12c6f06/payload-processing/0.log" Jul 20 18:45:18.763854 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:18.763824 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5fwpv_dfa09ac6-3677-47a8-9215-967b2edacfe0/main/0.log" Jul 20 18:45:19.137878 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:19.137851 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq8kjj_fbc13a29-690b-41af-bb91-4c4f3758f840/main/0.log" Jul 20 18:45:27.971414 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:27.971379 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5c4v2_f8584624-316d-4b6e-9463-3ead0ce5b732/global-pull-secret-syncer/0.log" Jul 20 18:45:28.168733 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:28.168700 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-llkf2_9b8386c3-4a92-46e9-9ac6-b93d336c0ca2/konnectivity-agent/0.log" Jul 20 18:45:28.194809 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:28.194772 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-135.ec2.internal_e6af33a66b3bbfc5fa2f8affa10cca68/haproxy/0.log" Jul 20 18:45:32.852190 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:32.852099 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg_0558c5bb-18b8-4534-9087-e203a0276770/extract/0.log" Jul 20 18:45:32.875833 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:32.875805 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg_0558c5bb-18b8-4534-9087-e203a0276770/util/0.log" Jul 20 18:45:32.913311 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:32.913282 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m27jg_0558c5bb-18b8-4534-9087-e203a0276770/pull/0.log" Jul 20 18:45:32.964983 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:32.964947 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx_9decc64d-d9d2-41cc-a07f-3b407a61738f/extract/0.log" Jul 20 18:45:32.996398 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:32.996374 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx_9decc64d-d9d2-41cc-a07f-3b407a61738f/util/0.log" Jul 20 18:45:33.025652 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.025629 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gbhgx_9decc64d-d9d2-41cc-a07f-3b407a61738f/pull/0.log" Jul 20 18:45:33.064796 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.064762 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw_4a45a589-a43d-4b2a-8eb6-b69c460e6f6e/extract/0.log" Jul 20 18:45:33.086822 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.086792 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw_4a45a589-a43d-4b2a-8eb6-b69c460e6f6e/util/0.log" Jul 20 18:45:33.108048 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.107974 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lgppw_4a45a589-a43d-4b2a-8eb6-b69c460e6f6e/pull/0.log" Jul 20 18:45:33.136567 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.136533 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z_d55eaab4-37b9-4792-9a21-8c3e77bf5710/extract/0.log" Jul 20 18:45:33.170314 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.170295 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z_d55eaab4-37b9-4792-9a21-8c3e77bf5710/util/0.log" Jul 20 18:45:33.191763 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.191726 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tvn5z_d55eaab4-37b9-4792-9a21-8c3e77bf5710/pull/0.log" Jul 20 18:45:33.811852 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.811820 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9f5557f5d-28kvz_f4415671-9428-4b45-8ba9-5092769a1ccb/authorino/0.log" Jul 20 18:45:33.841666 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.841642 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kmq6z_b2e119da-1f94-4777-9107-2f152558ab06/manager/0.log" Jul 20 18:45:33.869906 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.869880 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lq9vh_945777a1-35fb-4ae0-83f9-90584f9adaf7/manager/0.log" Jul 20 18:45:33.970505 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:33.970475 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lfw48_1ccf85ab-fb69-4e69-ba1c-65d079d9086b/registry-server/0.log" Jul 20 18:45:34.306369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:34.306342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4j4jh_99a3c160-632a-4a1b-8f76-c20917e9c112/manager/0.log" Jul 20 18:45:36.379925 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:36.379891 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-7lqxb_67719987-9554-49b7-ad4c-b68f29abfd2f/monitoring-plugin/0.log" Jul 20 18:45:36.531200 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:36.531158 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4jqt_2bfa4f73-8012-4ef9-8e5d-f99e067a6895/node-exporter/0.log" Jul 20 18:45:36.575822 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:36.575788 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4jqt_2bfa4f73-8012-4ef9-8e5d-f99e067a6895/kube-rbac-proxy/0.log" Jul 20 18:45:36.635372 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:36.635294 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4jqt_2bfa4f73-8012-4ef9-8e5d-f99e067a6895/init-textfile/0.log" Jul 20 18:45:37.117903 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.117853 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-hwmq4_3c94574e-4dc8-4b76-b552-2bc13e9d25c7/prometheus-operator-admission-webhook/0.log" Jul 20 18:45:37.149471 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.149443 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69d99cd586-tmfs6_b9b91dbb-0910-4e5c-95e1-f08a9e593748/telemeter-client/0.log" Jul 20 18:45:37.178019 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.177987 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69d99cd586-tmfs6_b9b91dbb-0910-4e5c-95e1-f08a9e593748/reload/0.log" Jul 20 18:45:37.214032 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.214005 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69d99cd586-tmfs6_b9b91dbb-0910-4e5c-95e1-f08a9e593748/kube-rbac-proxy/0.log" Jul 20 18:45:37.249487 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.249454 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f9875999-rrn98_043abbe2-ad48-4653-8c10-677fc1c69109/thanos-query/0.log" Jul 20 18:45:37.278515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.278490 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f9875999-rrn98_043abbe2-ad48-4653-8c10-677fc1c69109/kube-rbac-proxy-web/0.log" Jul 20 18:45:37.301695 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.301665 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f9875999-rrn98_043abbe2-ad48-4653-8c10-677fc1c69109/kube-rbac-proxy/0.log" Jul 20 18:45:37.326948 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.326925 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f9875999-rrn98_043abbe2-ad48-4653-8c10-677fc1c69109/prom-label-proxy/0.log" Jul 20 18:45:37.350861 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.350843 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f9875999-rrn98_043abbe2-ad48-4653-8c10-677fc1c69109/kube-rbac-proxy-rules/0.log" Jul 20 18:45:37.379780 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.379717 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f9875999-rrn98_043abbe2-ad48-4653-8c10-677fc1c69109/kube-rbac-proxy-metrics/0.log" Jul 20 18:45:37.904603 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.904571 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25"] Jul 20 18:45:37.905078 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.905061 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849ac704-cac1-4fa0-93e4-63a50c3c7ca3" containerName="revoke-keys" Jul 20 18:45:37.905078 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.905080 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="849ac704-cac1-4fa0-93e4-63a50c3c7ca3" containerName="revoke-keys" Jul 20 18:45:37.905226 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.905171 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="849ac704-cac1-4fa0-93e4-63a50c3c7ca3" containerName="revoke-keys" Jul 20 18:45:37.908342 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.908326 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:37.921677 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.921656 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5j77\"/\"default-dockercfg-sk2sz\"" Jul 20 18:45:37.921865 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.921682 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5j77\"/\"openshift-service-ca.crt\"" Jul 20 18:45:37.922533 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.922513 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5j77\"/\"kube-root-ca.crt\"" Jul 20 18:45:37.929497 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:37.929475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25"] Jul 20 18:45:38.039145 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.039107 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-lib-modules\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.039369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.039246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-podres\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.039369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.039273 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjfx5\" (UniqueName: \"kubernetes.io/projected/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-kube-api-access-gjfx5\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.039369 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.039329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-proc\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.039537 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.039417 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-sys\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140164 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140128 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-lib-modules\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140345 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140244 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-podres\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140345 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjfx5\" (UniqueName: \"kubernetes.io/projected/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-kube-api-access-gjfx5\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140345 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-proc\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140345 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140310 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-lib-modules\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140383 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-proc\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140395 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-podres\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140396 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-sys\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.140515 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.140438 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-sys\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.149675 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.149653 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjfx5\" (UniqueName: \"kubernetes.io/projected/52f4311c-4fa2-4751-acd3-09bd5ebcaf41-kube-api-access-gjfx5\") pod \"perf-node-gather-daemonset-bkh25\" (UID: \"52f4311c-4fa2-4751-acd3-09bd5ebcaf41\") " pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.217857 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.217784 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.365729 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.365699 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25"] Jul 20 18:45:38.366395 ip-10-0-133-135 kubenswrapper[2567]: W0720 18:45:38.366373 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod52f4311c_4fa2_4751_acd3_09bd5ebcaf41.slice/crio-6c18efb5364d7a551d18596b95c6b7bf7382e2725568078af9cd4630918e345c WatchSource:0}: Error finding container 6c18efb5364d7a551d18596b95c6b7bf7382e2725568078af9cd4630918e345c: Status 404 returned error can't find the container with id 6c18efb5364d7a551d18596b95c6b7bf7382e2725568078af9cd4630918e345c Jul 20 18:45:38.837606 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.837574 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" event={"ID":"52f4311c-4fa2-4751-acd3-09bd5ebcaf41","Type":"ContainerStarted","Data":"a70df77679da56a3ac594874b7bda191d11db3f035028e9aadca991821036a28"} Jul 20 18:45:38.837822 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.837633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" event={"ID":"52f4311c-4fa2-4751-acd3-09bd5ebcaf41","Type":"ContainerStarted","Data":"6c18efb5364d7a551d18596b95c6b7bf7382e2725568078af9cd4630918e345c"} Jul 20 18:45:38.837822 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.837691 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:38.858390 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:38.858340 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" podStartSLOduration=1.8583254359999999 podStartE2EDuration="1.858325436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:45:38.856250542 +0000 UTC m=+2967.984367742" watchObservedRunningTime="2026-07-20 18:45:38.858325436 +0000 UTC m=+2967.986442601" Jul 20 18:45:39.050104 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:39.050059 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/1.log" Jul 20 18:45:39.057599 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:39.057572 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rl452_e5783726-84ea-4afd-a948-32168ca72d7d/console-operator/2.log" Jul 20 18:45:39.570475 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:39.570443 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-544ddbdb4-clb26_3868656f-d6a9-4603-a945-b4dd0cfcaf1c/console/0.log" Jul 20 18:45:40.341828 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:40.341801 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-bpf82_ec33470c-21bf-40ba-9d16-45ca4f3bbcca/volume-data-source-validator/0.log" Jul 20 18:45:41.469204 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:41.469148 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvwbr_45dc5f22-b96b-457c-b060-a655c6b8adf0/dns/0.log" Jul 20 18:45:41.495451 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:41.495416 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvwbr_45dc5f22-b96b-457c-b060-a655c6b8adf0/kube-rbac-proxy/0.log" Jul 20 18:45:41.643867 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:41.643813 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vjrf4_02060614-726f-4116-8956-25ad464aa2c6/dns-node-resolver/0.log" Jul 20 18:45:42.205951 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:42.205910 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6989456fd7-mfs4w_c393437a-06b7-4663-a90f-ef4417c866d1/registry/0.log" Jul 20 18:45:42.287993 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:42.287964 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xrnks_a7f78e87-a5a8-4b46-9933-8aec39656f7c/node-ca/0.log" Jul 20 18:45:43.234322 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:43.234291 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4d7j47_57aadb03-24ae-414a-80d7-6aa445139129/istio-proxy/0.log" Jul 20 18:45:43.399310 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:43.399280 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75cc6b94bb-szq28_e5d8d836-6592-4308-ac78-e6bd720f66fd/kube-auth-proxy/0.log" Jul 20 18:45:43.462467 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:43.462435 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/0.log" Jul 20 18:45:43.511652 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:43.511570 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/1.log" Jul 20 18:45:43.553335 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:43.553307 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wp2ks_9b9575a6-2fca-43e5-b804-1e5e32f9dc68/payload-pre-processing/0.log" Jul 20 18:45:43.594037 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:43.594010 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5nm5f_94b2cfc1-12b2-4d23-b444-63ccc12c6f06/payload-processing/0.log" Jul 20 18:45:44.423919 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:44.423890 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bkx8q_5882388a-c3ea-48a4-bed7-db61cd700c12/serve-healthcheck-canary/0.log" Jul 20 18:45:44.853994 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:44.853967 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5j77/perf-node-gather-daemonset-bkh25" Jul 20 18:45:44.969454 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:44.969421 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-nkrsq_d275ccd4-1373-4b29-bb57-ca14e91580d9/insights-operator/0.log" Jul 20 18:45:44.970851 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:44.970823 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-nkrsq_d275ccd4-1373-4b29-bb57-ca14e91580d9/insights-operator/1.log" Jul 20 18:45:45.189146 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:45.189057 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ts7vs_ed285dc0-4058-4afa-a620-42799e4245ed/kube-rbac-proxy/0.log" Jul 20 18:45:45.214485 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:45.214460 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ts7vs_ed285dc0-4058-4afa-a620-42799e4245ed/exporter/0.log" Jul 20 18:45:45.239721 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:45.239694 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ts7vs_ed285dc0-4058-4afa-a620-42799e4245ed/extractor/0.log" Jul 20 18:45:47.741290 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:47.741263 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gx8bj_5d4341e7-78f1-442b-aabb-6b381486e766/manager/0.log" Jul 20 18:45:48.071187 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.071149 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2gdzv_0c757ec3-9d16-4264-97d2-4758e47d8703/manager/0.log" Jul 20 18:45:48.183390 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.183360 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4d7j47_57aadb03-24ae-414a-80d7-6aa445139129/istio-proxy/0.log" Jul 20 18:45:48.341165 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.341137 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75cc6b94bb-szq28_e5d8d836-6592-4308-ac78-e6bd720f66fd/kube-auth-proxy/0.log" Jul 20 18:45:48.384727 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.384702 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mwntz_dbd3ffc2-64b6-48dc-81a7-3ef3a0cdff2e/server/0.log" Jul 20 18:45:48.387857 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.387832 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/0.log" Jul 20 18:45:48.447832 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.447806 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/1.log" Jul 20 18:45:48.495447 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.495411 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wp2ks_9b9575a6-2fca-43e5-b804-1e5e32f9dc68/payload-pre-processing/0.log" Jul 20 18:45:48.515426 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.515396 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-drdk6_1b4cfa7a-f02a-4713-bc09-4e5f4bd44dd4/manager/0.log" Jul 20 18:45:48.532776 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.532745 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5nm5f_94b2cfc1-12b2-4d23-b444-63ccc12c6f06/payload-processing/0.log" Jul 20 18:45:48.715030 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.714948 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79747b4d4c-dqbms_e190c526-deae-442b-bb20-7edbafa87611/manager/0.log" Jul 20 18:45:48.746547 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:48.746518 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79747b4d4c-dqbms_e190c526-deae-442b-bb20-7edbafa87611/copy-manifests/0.log" Jul 20 18:45:49.828359 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:49.828309 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4d7j47_57aadb03-24ae-414a-80d7-6aa445139129/istio-proxy/0.log" Jul 20 18:45:49.969657 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:49.969619 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75cc6b94bb-szq28_e5d8d836-6592-4308-ac78-e6bd720f66fd/kube-auth-proxy/0.log" Jul 20 18:45:50.037560 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:50.037477 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/0.log" Jul 20 18:45:50.089720 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:50.089689 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m9ntd_8c5f0746-dcec-430b-b8ea-9ced3d2a43ca/istio-proxy/1.log" Jul 20 18:45:50.129138 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:50.129106 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wp2ks_9b9575a6-2fca-43e5-b804-1e5e32f9dc68/payload-pre-processing/0.log" Jul 20 18:45:50.175120 ip-10-0-133-135 kubenswrapper[2567]: I0720 18:45:50.175087 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-5nm5f_94b2cfc1-12b2-4d23-b444-63ccc12c6f06/payload-processing/0.log"