Jul 07 19:18:54.253237 ip-10-0-130-242 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 19:18:54.253249 ip-10-0-130-242 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 19:18:54.253259 ip-10-0-130-242 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 19:18:54.253572 ip-10-0-130-242 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 19:19:04.322893 ip-10-0-130-242 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 19:19:04.322915 ip-10-0-130-242 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a159cd138fcf481dbbc8c23190835600 -- Jul 07 19:20:52.646609 ip-10-0-130-242 systemd[1]: Starting Kubernetes Kubelet... Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.070963 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077468 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077483 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077487 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077492 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077496 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077501 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077505 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:20:53.136581 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077513 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077518 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077522 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077525 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077530 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077536 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077543 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077547 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077552 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077556 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077560 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077564 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077567 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077571 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077575 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077579 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077583 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077587 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077591 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:20:53.137367 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077595 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077599 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077605 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077610 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077615 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077619 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077624 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077628 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077632 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077636 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077640 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077644 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077656 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077660 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077664 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077668 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077672 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077675 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077679 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:20:53.137889 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077683 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077686 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077690 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077694 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077697 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077701 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077705 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077708 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077712 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077716 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077720 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077724 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077728 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077731 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077735 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077748 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077752 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077756 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077760 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077763 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:20:53.138532 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077767 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077771 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077774 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077778 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077782 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077786 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077797 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077801 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077804 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077811 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077816 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077821 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077825 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077828 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077832 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077836 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077840 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077844 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077848 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077852 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:20:53.139215 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077855 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077860 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077863 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077867 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077871 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077875 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077879 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077883 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077886 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077890 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077894 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077898 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077902 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.077905 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.078998 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079007 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079011 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079015 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079021 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079028 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:20:53.139848 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079032 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079036 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079040 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079044 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079048 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079051 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079055 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079060 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079064 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079068 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079072 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079076 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079080 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079085 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079089 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079093 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079097 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079102 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079106 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079110 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:20:53.140503 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079113 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079117 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079121 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079125 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079129 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079133 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079137 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079141 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079145 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079149 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079154 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079157 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079162 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079166 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079169 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079173 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079176 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079180 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079183 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079187 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:20:53.141178 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079191 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079194 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079198 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079202 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079206 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079210 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079214 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079217 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079221 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079225 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079235 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079240 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079245 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079250 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079254 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079258 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079262 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079267 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079270 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:20:53.141996 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079274 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079278 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079282 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079286 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079291 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079296 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079300 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079304 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079308 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079311 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079332 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079336 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079340 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079344 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079348 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079352 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079355 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079360 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079366 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:20:53.142812 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079370 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079374 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079379 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079382 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079387 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079390 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079394 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079398 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079401 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079405 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079409 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079412 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079416 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079419 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.079423 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080850 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080858 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080863 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080868 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080873 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:20:53.143637 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080877 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080881 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080885 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080889 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080893 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080896 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080905 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080909 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080912 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080916 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080920 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080924 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080927 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080931 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080936 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080939 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080943 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080947 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080951 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080955 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:20:53.144474 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080959 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080963 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080966 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080970 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080977 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080981 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080984 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080990 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080995 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.080999 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081004 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081009 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081013 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081016 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081020 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081024 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081028 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081031 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081035 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:20:53.145267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081040 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081044 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081049 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081053 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081057 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081061 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081065 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081069 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081075 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081080 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081084 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081088 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081091 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081095 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081098 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081102 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081106 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081110 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081115 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081119 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:20:53.145840 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081122 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081126 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081130 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081134 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081138 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081142 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081150 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081154 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081158 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081162 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081166 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081170 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081174 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081177 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081181 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081184 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081190 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081193 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081197 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081201 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:20:53.146480 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081205 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081209 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081212 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081217 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081220 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081223 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081227 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081231 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081234 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081239 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081243 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081247 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081251 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081255 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.081258 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081360 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081369 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081383 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081391 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081398 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081402 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 19:20:53.147507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081408 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081414 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081419 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081424 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081429 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081433 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081438 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081442 2561 flags.go:64] FLAG: --cgroup-root="" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081447 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081451 2561 flags.go:64] FLAG: --client-ca-file="" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081455 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081459 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081483 2561 flags.go:64] FLAG: --cluster-domain="" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081488 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081494 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081499 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081504 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081510 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081514 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081519 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081525 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081529 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081533 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081537 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 19:20:53.148539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081542 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081546 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081551 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081556 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081560 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081565 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081570 2561 flags.go:64] FLAG: --enable-server="true" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081575 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081581 2561 flags.go:64] FLAG: --event-burst="100" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081585 2561 flags.go:64] FLAG: --event-qps="50" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081590 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081594 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081598 2561 flags.go:64] FLAG: --eviction-hard="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081603 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081608 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081613 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081617 2561 flags.go:64] FLAG: --eviction-soft="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081622 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081626 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081630 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081634 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081638 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081643 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081647 2561 flags.go:64] FLAG: --feature-gates="" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081652 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 19:20:53.149513 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081656 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081660 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081665 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081670 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081675 2561 flags.go:64] FLAG: --help="false" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081679 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-130-242.ec2.internal" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081684 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081688 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081692 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081697 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081702 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081706 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081711 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081716 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081720 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081725 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081730 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081734 2561 flags.go:64] FLAG: --kube-reserved="" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081739 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081743 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081748 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081751 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081756 2561 flags.go:64] FLAG: --lock-file="" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081760 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 19:20:53.150530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081763 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081767 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081775 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081779 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081784 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081787 2561 flags.go:64] FLAG: --logging-format="text" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081792 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081797 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081801 2561 flags.go:64] FLAG: --manifest-url="" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081805 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081810 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081816 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081822 2561 flags.go:64] FLAG: --max-pods="110" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081826 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081831 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081835 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081838 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081843 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081847 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081852 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081862 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081867 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081871 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 19:20:53.151565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081876 2561 flags.go:64] FLAG: --pod-cidr="" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081881 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081888 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081893 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081897 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081902 2561 flags.go:64] FLAG: --port="10250" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081906 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081910 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e994473450daf2f2" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081915 2561 flags.go:64] FLAG: --qos-reserved="" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081919 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081924 2561 flags.go:64] FLAG: --register-node="true" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081928 2561 flags.go:64] FLAG: --register-with-taints="" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081933 2561 flags.go:64] FLAG: --registry-burst="10" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081937 2561 flags.go:64] FLAG: --registry-qps="5" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081941 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081945 2561 flags.go:64] FLAG: --reserved-memory="" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081951 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081955 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081960 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081964 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081968 2561 flags.go:64] FLAG: --runonce="false" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081973 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081978 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081983 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081987 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 19:20:53.152204 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081991 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081995 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.081999 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082004 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082009 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082013 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082018 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082022 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082027 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082032 2561 flags.go:64] FLAG: --system-cgroups="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082036 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082044 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082048 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082052 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082058 2561 flags.go:64] FLAG: --tls-min-version="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082063 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082067 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082071 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082075 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082080 2561 flags.go:64] FLAG: --v="2" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082084 2561 flags.go:64] FLAG: --version="false" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082090 2561 flags.go:64] FLAG: --vmodule="" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082094 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.082099 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082253 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:20:53.257050 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082259 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:20:53.153396 ip-10-0-130-242 systemd[1]: Started Kubernetes Kubelet. Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082263 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082268 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082274 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082278 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082282 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082286 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082289 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082293 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082297 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082301 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082307 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082311 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082327 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082331 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082335 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082339 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082343 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082346 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082350 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:20:53.257918 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082354 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082358 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082361 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082365 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082369 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082372 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082376 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082379 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082383 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082387 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082390 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082394 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082398 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082401 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082406 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082411 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082415 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082419 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082422 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:20:53.258696 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082428 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082433 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082437 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082441 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082447 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082450 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082454 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082458 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082461 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082466 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082469 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082479 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082500 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082504 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082509 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082514 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082518 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082522 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082526 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082530 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:20:53.259441 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082534 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082538 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082543 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082547 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082551 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082554 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082558 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082562 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082567 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082571 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082575 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082579 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082583 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082586 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082590 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082593 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082602 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082608 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082613 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:20:53.260462 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082617 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082621 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082625 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082629 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082634 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082638 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082641 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082645 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082649 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082652 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082656 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082660 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082664 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082667 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082671 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082675 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082678 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082682 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082686 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:20:53.261055 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.082690 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.083360 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.090115 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.090130 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.090149 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.090154 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090207 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090212 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090215 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090219 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090222 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090225 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090227 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090230 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:20:53.261721 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090233 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090236 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090239 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090241 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090245 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090248 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090251 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090254 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090257 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090260 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090263 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090266 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090269 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090272 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090274 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090277 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090279 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090282 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090284 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090287 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:20:53.262155 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090289 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090292 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090295 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090298 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090300 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090303 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090306 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090308 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090311 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090328 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090332 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090334 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090338 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090341 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090343 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090346 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090349 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090353 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090357 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090360 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:20:53.262852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090363 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090365 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090368 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090370 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090373 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090375 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090378 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090380 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090384 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090388 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090390 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090393 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090396 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090399 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090402 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090408 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090411 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090414 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090416 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090419 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:20:53.415970 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090421 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090424 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090427 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090429 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090431 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090434 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090437 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090439 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090442 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090446 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090449 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090452 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090456 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090458 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090461 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090464 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090467 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090469 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090472 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:20:53.416867 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090474 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090476 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090479 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090482 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090484 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090487 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090489 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090492 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090496 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090499 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090501 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090504 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.090509 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090618 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090622 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 19:20:53.417648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090625 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090628 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090631 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090634 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090637 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090639 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090642 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090644 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090647 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090650 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090653 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090656 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090659 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090662 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090664 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090667 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090669 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090672 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090674 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090677 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 19:20:53.418727 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090679 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090682 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090684 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090686 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090691 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090693 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090697 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090700 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090702 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090705 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090708 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090710 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090713 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090715 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090718 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090721 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090723 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090726 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090728 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 19:20:53.419380 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090732 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090735 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090738 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090741 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090744 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090747 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090750 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090752 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090755 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090757 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090760 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090762 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090765 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090767 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090770 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090773 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090776 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090780 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090782 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090787 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 19:20:53.419978 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090789 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090792 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090795 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090797 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090800 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090802 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090805 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090807 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090810 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090812 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090814 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090817 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090819 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090822 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090824 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090827 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090841 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090844 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090847 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090849 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 19:20:53.420725 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090852 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090855 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090859 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090862 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090865 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090867 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090870 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090872 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090875 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090879 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090882 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090887 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090889 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090893 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090896 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090899 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090902 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 19:20:53.421297 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:53.090904 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.090909 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.091006 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.093950 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.095349 2561 server.go:1019] "Starting client certificate rotation" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.095477 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.095507 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.098840 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.104996 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.115846 2561 log.go:25] "Validated CRI v1 runtime API" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.117893 2561 log.go:25] "Validated CRI v1 image API" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.118589 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 19:20:53.421937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.121045 2561 fs.go:135] Filesystem UUIDs: map[5b8d59f5-f187-4b42-a1b2-883dc165366d:/dev/nvme0n1p4 78b02475-dc74-41d1-860a-72c3f1b404bc:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 07 19:20:53.422349 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.121067 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 19:20:53.422349 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.124756 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.125467 2561 manager.go:216] Machine: {Timestamp:2026-07-07 19:20:53.124981188 +0000 UTC m=+0.341921147 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100841 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d28e5c15ed63df4eb0e65a2e0e7e6 SystemUUID:ec2d28e5-c15e-d63d-f4eb-0e65a2e0e7e6 BootID:a159cd13-8fcf-481d-bbc8-c23190835600 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:25:cd:80:2f:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:25:cd:80:2f:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:0c:03:69:03:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.125564 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.125667 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.128999 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.129026 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-242.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.130249 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.130260 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.130267 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.131009 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.132364 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.137847 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.137958 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.137985 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.138003 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.138014 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.138023 2561 kubelet.go:398] "Adding apiserver pod source" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.138073 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 19:20:53.422629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.138540 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.138574 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.141123 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.143102 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.143141 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144834 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144858 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144869 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144878 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144890 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144899 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144912 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144920 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144930 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144939 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144953 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.144962 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.146347 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.146362 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.150153 2561 server.go:1268] "Started kubelet" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.150229 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.152717 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.153423 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.153930 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-242.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.154822 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.154821 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.155064 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.157170 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.158393 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-242.ec2.internal.18c0183175f42e6b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-242.ec2.internal,UID:ip-10-0-130-242.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-242.ec2.internal,},FirstTimestamp:2026-07-07 19:20:53.150117483 +0000 UTC m=+0.367057704,LastTimestamp:2026-07-07 19:20:53.150117483 +0000 UTC m=+0.367057704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-242.ec2.internal,}" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.164429 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.167309 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.167327 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.167387 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.167902 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.167922 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.167902 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168030 2561 factory.go:55] Registering systemd factory Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168045 2561 factory.go:223] Registration of the systemd container factory successfully Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168062 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168070 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168273 2561 factory.go:153] Registering CRI-O factory Jul 07 19:20:53.423158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168285 2561 factory.go:223] Registration of the crio container factory successfully Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168351 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168374 2561 factory.go:103] Registering Raw factory Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168385 2561 manager.go:1196] Started watching for new ooms in manager Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.168787 2561 manager.go:349] Starting recovery of all containers Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.169812 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.169919 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.177046 2561 manager.go:354] Recovery completed Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.179700 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8zpn8" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.182557 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184396 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184423 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184433 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184917 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184927 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184940 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.184928 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.185597 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.185617 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.185657 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.185657 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8zpn8" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.186362 2561 policy_none.go:49] "None policy: Start" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.186373 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.186383 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.187909 2561 policy_none.go:47] "Start" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.237750 2561 manager.go:337] "Starting Device Plugin manager" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.237896 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.237909 2561 server.go:87] "Starting device plugin registration server" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.238131 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.238142 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.238230 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.238300 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.238308 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.238793 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.238841 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.298582 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.301533 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.301574 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.301599 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.301637 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.304124 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.338244 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.339041 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.339066 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.339078 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.339107 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-242.ec2.internal" Jul 07 19:20:53.424308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.348301 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-242.ec2.internal" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.348333 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-242.ec2.internal\": node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.375210 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.402425 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal"] Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.402493 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.404197 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.404227 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.404241 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406058 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406207 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406234 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406822 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406860 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406870 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406823 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406924 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.406934 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.408823 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.408847 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.409465 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.409493 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 19:20:53.425555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.409508 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 07 19:20:54.883582 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.455029 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-242.ec2.internal\" not found" node="ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.883582 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.459446 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-242.ec2.internal\" not found" node="ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.883582 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.469478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d37125de82a4e9d646d5bc6cbdf2098-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"4d37125de82a4e9d646d5bc6cbdf2098\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.883582 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.469503 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d37125de82a4e9d646d5bc6cbdf2098-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"4d37125de82a4e9d646d5bc6cbdf2098\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.883582 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:53.469521 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1d6e4c9d780b8824beae118385da5e5-config\") pod \"kube-apiserver-proxy-ip-10-0-130-242.ec2.internal\" (UID: \"e1d6e4c9d780b8824beae118385da5e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.883582 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:53.475563 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:54.884887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:54.884764 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d37125de82a4e9d646d5bc6cbdf2098-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"4d37125de82a4e9d646d5bc6cbdf2098\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.884887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:54.884791 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d37125de82a4e9d646d5bc6cbdf2098-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"4d37125de82a4e9d646d5bc6cbdf2098\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.884887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:54.884829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d37125de82a4e9d646d5bc6cbdf2098-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"4d37125de82a4e9d646d5bc6cbdf2098\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.884887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:54.884850 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1d6e4c9d780b8824beae118385da5e5-config\") pod \"kube-apiserver-proxy-ip-10-0-130-242.ec2.internal\" (UID: \"e1d6e4c9d780b8824beae118385da5e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.884887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:54.884877 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1d6e4c9d780b8824beae118385da5e5-config\") pod \"kube-apiserver-proxy-ip-10-0-130-242.ec2.internal\" (UID: \"e1d6e4c9d780b8824beae118385da5e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 07 19:20:54.885210 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:54.884909 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d37125de82a4e9d646d5bc6cbdf2098-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"4d37125de82a4e9d646d5bc6cbdf2098\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:55.030136 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:54.890844 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:55.030136 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:54.991507 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:55.137295 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.073704 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:55.137295 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.077687 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 07 19:20:55.137295 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.091813 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:55.276970 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.192352 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:55.401657 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.292774 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:55.401657 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.295500 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:20:55.401657 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.393313 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.408191 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.410193 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.410292 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.410346 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.453484 2561 apiserver.go:52] "Watching apiserver" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.473724 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.474588 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-pkr65","openshift-ovn-kubernetes/ovnkube-node-nrh2c","kube-system/global-pull-secret-syncer-j474f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt","openshift-cluster-node-tuning-operator/tuned-lqcxh","openshift-dns/node-resolver-fm4q2","openshift-multus/network-metrics-daemon-cvqcz","openshift-network-diagnostics/network-check-target-88vw9","openshift-network-operator/iptables-alerter-b4lgl","kube-system/konnectivity-agent-d7b7z","openshift-image-registry/node-ca-4w272","openshift-multus/multus-additional-cni-plugins-4wnkk"] Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.476458 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.478229 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.479693 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.481240 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.481747 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.481943 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.481949 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.482403 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.482463 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.482492 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.482498 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.482999 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483414 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483687 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483721 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rb68r\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483737 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.483747 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483689 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483802 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.483739 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.484058 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.485095 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.486615 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.526358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.486686 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487701 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-etc-selinux\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487731 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-kubernetes\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487752 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-tuned\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-socket-dir-parent\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-kubelet\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487822 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-env-overrides\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487842 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovnkube-script-lib\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysctl-conf\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487884 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqtg\" (UniqueName: \"kubernetes.io/projected/5e094b07-2084-42a5-9e20-9852d2eea7c5-kube-api-access-dtqtg\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487905 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-cni-bin\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-log-socket\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487951 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/995f0143-41e0-4437-bc4d-4ceaab716052-multus-daemon-config\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.487990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-multus-certs\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488013 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-ovn\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488035 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-device-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.530795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488054 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-sys\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-var-lib-kubelet\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-etc-kubernetes\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-registration-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-run\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgf88\" (UniqueName: \"kubernetes.io/projected/92f94e51-0ce7-4298-95ea-b6df5e03d944-kube-api-access-jgf88\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488194 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-cnibin\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488213 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-k8s-cni-cncf-io\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488227 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-run-netns\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488240 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovnkube-config\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488242 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovn-node-metrics-cert\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-var-lib-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488332 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-cni-netd\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488352 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-cni-multus\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488400 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4x54\" (UniqueName: \"kubernetes.io/projected/995f0143-41e0-4437-bc4d-4ceaab716052-kube-api-access-g4x54\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.531974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysconfig\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488457 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-lib-modules\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/995f0143-41e0-4437-bc4d-4ceaab716052-cni-binary-copy\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-kubelet\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-sys-fs\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-system-cni-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-cni-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488613 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488643 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjh2r\" (UniqueName: \"kubernetes.io/projected/2be57a06-c44d-4ef3-9e28-a0384c082bdc-kube-api-access-bjh2r\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488658 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97b5\" (UniqueName: \"kubernetes.io/projected/16f7b774-6866-4593-babd-1deaa73c8dd6-kube-api-access-v97b5\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-cni-bin\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-socket-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-host\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-slash\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488728 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-systemd\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.536423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488749 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-etc-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488766 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-os-release\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-hostroot\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488792 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-conf-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488833 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-systemd-units\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-node-log\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e094b07-2084-42a5-9e20-9852d2eea7c5-tmp\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488915 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488943 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-modprobe-d\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysctl-d\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488974 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-netns\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.488987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-systemd\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.489978 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.490038 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.491515 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.491558 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.492634 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.494220 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.537192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.494220 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.494445 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g7dp9\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.494460 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.494673 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k5shc\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.494679 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.496296 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499061 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k6kcl\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499135 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499147 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gvw8z\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499187 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499198 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499242 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zmxrp\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499265 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499199 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4rtlw\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499360 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499396 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2bdjg\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499544 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499674 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499703 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499757 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499788 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9j6bp\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.499926 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.502685 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.502955 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 19:15:54 +0000 UTC" deadline="2028-01-23 00:24:02.878703025 +0000 UTC" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.502996 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13541h3m7.375709105s" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.509080 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal"] Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.509580 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.509648 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.520600 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal"] Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.521151 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 19:20:55.540916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.533332 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbg2v" Jul 07 19:20:55.544536 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.541942 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbg2v" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.583666 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589564 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589586 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad798b99-e8b2-439e-ba56-c9805dfacd9d-tmp-dir\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6933ad4a-603f-45fc-8821-14edc1cda57e-konnectivity-ca\") pod \"konnectivity-agent-d7b7z\" (UID: \"6933ad4a-603f-45fc-8821-14edc1cda57e\") " pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589625 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndpf5\" (UniqueName: \"kubernetes.io/projected/0964a1be-fa18-4329-a4c6-e93935334f52-kube-api-access-ndpf5\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589641 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589661 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-modprobe-d\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589751 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysctl-d\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589843 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-modprobe-d\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589874 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysctl-d\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rf9q\" (UniqueName: \"kubernetes.io/projected/338da55c-a41b-4808-8c26-859f730f4ed5-kube-api-access-5rf9q\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589930 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-netns\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589956 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-systemd\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.589989 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-systemd\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad798b99-e8b2-439e-ba56-c9805dfacd9d-hosts-file\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590009 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-netns\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590035 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-os-release\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590066 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfjbg\" (UniqueName: \"kubernetes.io/projected/dab8ec61-668d-45b0-99d3-aa501eb38b63-kube-api-access-bfjbg\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-etc-selinux\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-kubernetes\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590134 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-tuned\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590167 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-socket-dir-parent\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590188 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-kubelet\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-env-overrides\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590217 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-etc-selinux\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590220 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-kubernetes\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590231 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovnkube-script-lib\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590250 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-kubelet\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysctl-conf\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590274 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-socket-dir-parent\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.656993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590287 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqtg\" (UniqueName: \"kubernetes.io/projected/5e094b07-2084-42a5-9e20-9852d2eea7c5-kube-api-access-dtqtg\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-system-cni-dir\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590335 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590341 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysctl-conf\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-cni-bin\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590391 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-log-socket\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590409 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-cni-bin\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/995f0143-41e0-4437-bc4d-4ceaab716052-multus-daemon-config\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590437 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-log-socket\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590443 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-multus-certs\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590475 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-ovn\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590492 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-device-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590487 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-multus-certs\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590510 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-sys\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590533 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-var-lib-kubelet\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590542 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-ovn\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590556 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-device-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.657705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590569 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-sys\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590559 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6933ad4a-603f-45fc-8821-14edc1cda57e-agent-certs\") pod \"konnectivity-agent-d7b7z\" (UID: \"6933ad4a-603f-45fc-8821-14edc1cda57e\") " pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590597 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-var-lib-kubelet\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590604 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b23202de-78de-407f-b354-2c83ecd07eed-kubelet-config\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-etc-kubernetes\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-registration-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-run\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590681 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-etc-kubernetes\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgf88\" (UniqueName: \"kubernetes.io/projected/92f94e51-0ce7-4298-95ea-b6df5e03d944-kube-api-access-jgf88\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590720 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-registration-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590744 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-run\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0964a1be-fa18-4329-a4c6-e93935334f52-serviceca\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b23202de-78de-407f-b354-2c83ecd07eed-dbus\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-cnibin\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-k8s-cni-cncf-io\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590881 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-cnibin\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-run-netns\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590904 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-run-k8s-cni-cncf-io\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovnkube-config\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovn-node-metrics-cert\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590953 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-var-lib-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.590936 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-run-netns\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591021 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-var-lib-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-cni-netd\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591085 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591093 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591112 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-cni-multus\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591120 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-cni-netd\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591133 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4x54\" (UniqueName: \"kubernetes.io/projected/995f0143-41e0-4437-bc4d-4ceaab716052-kube-api-access-g4x54\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591148 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-cni-multus\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591150 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysconfig\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.661992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591171 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-lib-modules\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0964a1be-fa18-4329-a4c6-e93935334f52-host\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591205 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-sysconfig\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dab8ec61-668d-45b0-99d3-aa501eb38b63-host-slash\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/995f0143-41e0-4437-bc4d-4ceaab716052-cni-binary-copy\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591312 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-lib-modules\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591335 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-kubelet\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591357 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-host-var-lib-kubelet\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591374 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-sys-fs\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591410 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-sys-fs\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-system-cni-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-cni-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591498 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-system-cni-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjh2r\" (UniqueName: \"kubernetes.io/projected/2be57a06-c44d-4ef3-9e28-a0384c082bdc-kube-api-access-bjh2r\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97b5\" (UniqueName: \"kubernetes.io/projected/16f7b774-6866-4593-babd-1deaa73c8dd6-kube-api-access-v97b5\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.665591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591545 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591569 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-cni-bin\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591590 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-socket-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591610 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-cni-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591617 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-host\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-cnibin\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dab8ec61-668d-45b0-99d3-aa501eb38b63-iptables-alerter-script\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-slash\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.591524 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcllk\" (UniqueName: \"kubernetes.io/projected/ad798b99-e8b2-439e-ba56-c9805dfacd9d-kube-api-access-tcllk\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-systemd\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-etc-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.591756 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:20:56.091734513 +0000 UTC m=+1.993613030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-os-release\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-hostroot\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591821 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-cni-bin\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-conf-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.666626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591869 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-systemd-units\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591884 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e094b07-2084-42a5-9e20-9852d2eea7c5-host\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591891 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-node-log\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591913 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-run-systemd\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591919 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e094b07-2084-42a5-9e20-9852d2eea7c5-tmp\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591919 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-os-release\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591941 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-host-slash\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-etc-openvswitch\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591961 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-hostroot\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591994 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-node-log\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.591998 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be57a06-c44d-4ef3-9e28-a0384c082bdc-systemd-units\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.592064 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/995f0143-41e0-4437-bc4d-4ceaab716052-multus-conf-dir\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.592492 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/16f7b774-6866-4593-babd-1deaa73c8dd6-socket-dir\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.593416 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.594369 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/995f0143-41e0-4437-bc4d-4ceaab716052-multus-daemon-config\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.594374 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/995f0143-41e0-4437-bc4d-4ceaab716052-cni-binary-copy\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.594512 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-env-overrides\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.594859 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovnkube-script-lib\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.594912 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovnkube-config\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.766842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.596600 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e094b07-2084-42a5-9e20-9852d2eea7c5-etc-tuned\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.596616 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be57a06-c44d-4ef3-9e28-a0384c082bdc-ovn-node-metrics-cert\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.596660 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e094b07-2084-42a5-9e20-9852d2eea7c5-tmp\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.601987 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4x54\" (UniqueName: \"kubernetes.io/projected/995f0143-41e0-4437-bc4d-4ceaab716052-kube-api-access-g4x54\") pod \"multus-pkr65\" (UID: \"995f0143-41e0-4437-bc4d-4ceaab716052\") " pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.607430 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.607445 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.607454 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.607493 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:20:56.107482858 +0000 UTC m=+2.009361371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.609226 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97b5\" (UniqueName: \"kubernetes.io/projected/16f7b774-6866-4593-babd-1deaa73c8dd6-kube-api-access-v97b5\") pod \"aws-ebs-csi-driver-node-kbxvt\" (UID: \"16f7b774-6866-4593-babd-1deaa73c8dd6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.609732 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgf88\" (UniqueName: \"kubernetes.io/projected/92f94e51-0ce7-4298-95ea-b6df5e03d944-kube-api-access-jgf88\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.609827 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqtg\" (UniqueName: \"kubernetes.io/projected/5e094b07-2084-42a5-9e20-9852d2eea7c5-kube-api-access-dtqtg\") pod \"tuned-lqcxh\" (UID: \"5e094b07-2084-42a5-9e20-9852d2eea7c5\") " pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.613104 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjh2r\" (UniqueName: \"kubernetes.io/projected/2be57a06-c44d-4ef3-9e28-a0384c082bdc-kube-api-access-bjh2r\") pod \"ovnkube-node-nrh2c\" (UID: \"2be57a06-c44d-4ef3-9e28-a0384c082bdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-cnibin\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dab8ec61-668d-45b0-99d3-aa501eb38b63-iptables-alerter-script\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcllk\" (UniqueName: \"kubernetes.io/projected/ad798b99-e8b2-439e-ba56-c9805dfacd9d-kube-api-access-tcllk\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692421 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-cnibin\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad798b99-e8b2-439e-ba56-c9805dfacd9d-tmp-dir\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.767878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692480 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6933ad4a-603f-45fc-8821-14edc1cda57e-konnectivity-ca\") pod \"konnectivity-agent-d7b7z\" (UID: \"6933ad4a-603f-45fc-8821-14edc1cda57e\") " pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndpf5\" (UniqueName: \"kubernetes.io/projected/0964a1be-fa18-4329-a4c6-e93935334f52-kube-api-access-ndpf5\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rf9q\" (UniqueName: \"kubernetes.io/projected/338da55c-a41b-4808-8c26-859f730f4ed5-kube-api-access-5rf9q\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad798b99-e8b2-439e-ba56-c9805dfacd9d-hosts-file\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692674 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-os-release\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfjbg\" (UniqueName: \"kubernetes.io/projected/dab8ec61-668d-45b0-99d3-aa501eb38b63-kube-api-access-bfjbg\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692726 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692756 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-system-cni-dir\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692773 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad798b99-e8b2-439e-ba56-c9805dfacd9d-hosts-file\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692801 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-os-release\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692821 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6933ad4a-603f-45fc-8821-14edc1cda57e-agent-certs\") pod \"konnectivity-agent-d7b7z\" (UID: \"6933ad4a-603f-45fc-8821-14edc1cda57e\") " pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692920 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dab8ec61-668d-45b0-99d3-aa501eb38b63-iptables-alerter-script\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692931 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad798b99-e8b2-439e-ba56-c9805dfacd9d-tmp-dir\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692972 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b23202de-78de-407f-b354-2c83ecd07eed-kubelet-config\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.692986 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-system-cni-dir\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693001 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0964a1be-fa18-4329-a4c6-e93935334f52-serviceca\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.777319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693021 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b23202de-78de-407f-b354-2c83ecd07eed-dbus\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693032 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b23202de-78de-407f-b354-2c83ecd07eed-kubelet-config\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693050 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693079 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693139 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0964a1be-fa18-4329-a4c6-e93935334f52-host\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.693148 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dab8ec61-668d-45b0-99d3-aa501eb38b63-host-slash\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:55.693224 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:20:56.193206458 +0000 UTC m=+2.095084975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693246 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6933ad4a-603f-45fc-8821-14edc1cda57e-konnectivity-ca\") pod \"konnectivity-agent-d7b7z\" (UID: \"6933ad4a-603f-45fc-8821-14edc1cda57e\") " pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b23202de-78de-407f-b354-2c83ecd07eed-dbus\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693366 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0964a1be-fa18-4329-a4c6-e93935334f52-host\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693370 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/338da55c-a41b-4808-8c26-859f730f4ed5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693426 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dab8ec61-668d-45b0-99d3-aa501eb38b63-host-slash\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693473 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693493 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693573 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0964a1be-fa18-4329-a4c6-e93935334f52-serviceca\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.778786 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.693645 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/338da55c-a41b-4808-8c26-859f730f4ed5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.779722 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.694993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6933ad4a-603f-45fc-8821-14edc1cda57e-agent-certs\") pod \"konnectivity-agent-d7b7z\" (UID: \"6933ad4a-603f-45fc-8821-14edc1cda57e\") " pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:55.779722 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.701836 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rf9q\" (UniqueName: \"kubernetes.io/projected/338da55c-a41b-4808-8c26-859f730f4ed5-kube-api-access-5rf9q\") pod \"multus-additional-cni-plugins-4wnkk\" (UID: \"338da55c-a41b-4808-8c26-859f730f4ed5\") " pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:55.779722 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.701984 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcllk\" (UniqueName: \"kubernetes.io/projected/ad798b99-e8b2-439e-ba56-c9805dfacd9d-kube-api-access-tcllk\") pod \"node-resolver-fm4q2\" (UID: \"ad798b99-e8b2-439e-ba56-c9805dfacd9d\") " pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:55.779722 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.702277 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndpf5\" (UniqueName: \"kubernetes.io/projected/0964a1be-fa18-4329-a4c6-e93935334f52-kube-api-access-ndpf5\") pod \"node-ca-4w272\" (UID: \"0964a1be-fa18-4329-a4c6-e93935334f52\") " pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:55.779722 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.702337 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfjbg\" (UniqueName: \"kubernetes.io/projected/dab8ec61-668d-45b0-99d3-aa501eb38b63-kube-api-access-bfjbg\") pod \"iptables-alerter-b4lgl\" (UID: \"dab8ec61-668d-45b0-99d3-aa501eb38b63\") " pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:55.937956 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.937934 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pkr65" Jul 07 19:20:55.945139 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.945119 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:20:55.973959 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.973943 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" Jul 07 19:20:55.992112 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:55.992091 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" Jul 07 19:20:56.000641 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.000625 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b4lgl" Jul 07 19:20:56.006121 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.006105 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:20:56.030954 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.030931 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4w272" Jul 07 19:20:56.045104 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.045086 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:20:56.049665 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.049638 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0964a1be_fa18_4329_a4c6_e93935334f52.slice/crio-4988cc2b182db265670960016378df7274ae8aab3f3201f5af073fdad9244038 WatchSource:0}: Error finding container 4988cc2b182db265670960016378df7274ae8aab3f3201f5af073fdad9244038: Status 404 returned error can't find the container with id 4988cc2b182db265670960016378df7274ae8aab3f3201f5af073fdad9244038 Jul 07 19:20:56.049852 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.049832 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d37125de82a4e9d646d5bc6cbdf2098.slice/crio-bad73e2fa1f58d7e917792b647924dee598c80cdba758d7a07f23504bca1bbf2 WatchSource:0}: Error finding container bad73e2fa1f58d7e917792b647924dee598c80cdba758d7a07f23504bca1bbf2: Status 404 returned error can't find the container with id bad73e2fa1f58d7e917792b647924dee598c80cdba758d7a07f23504bca1bbf2 Jul 07 19:20:56.050057 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.050038 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d6e4c9d780b8824beae118385da5e5.slice/crio-50042c936aa683744708185dea527094eae30d9f08773770242737547f216d19 WatchSource:0}: Error finding container 50042c936aa683744708185dea527094eae30d9f08773770242737547f216d19: Status 404 returned error can't find the container with id 50042c936aa683744708185dea527094eae30d9f08773770242737547f216d19 Jul 07 19:20:56.054088 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.054075 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:20:56.057314 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.057298 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" Jul 07 19:20:56.062154 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.062139 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fm4q2" Jul 07 19:20:56.065742 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.065721 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338da55c_a41b_4808_8c26_859f730f4ed5.slice/crio-6a2e8709b3f96a7021faf7caa5c7300557fb557c1e29cb66feaa8d9571fe01f0 WatchSource:0}: Error finding container 6a2e8709b3f96a7021faf7caa5c7300557fb557c1e29cb66feaa8d9571fe01f0: Status 404 returned error can't find the container with id 6a2e8709b3f96a7021faf7caa5c7300557fb557c1e29cb66feaa8d9571fe01f0 Jul 07 19:20:56.070965 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.070945 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad798b99_e8b2_439e_ba56_c9805dfacd9d.slice/crio-28b9603f8ee6d90e4a0d820a8da8bf2f402c68cb2156f633d18f1a4813abba1f WatchSource:0}: Error finding container 28b9603f8ee6d90e4a0d820a8da8bf2f402c68cb2156f633d18f1a4813abba1f: Status 404 returned error can't find the container with id 28b9603f8ee6d90e4a0d820a8da8bf2f402c68cb2156f633d18f1a4813abba1f Jul 07 19:20:56.095626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.095606 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:56.095781 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.095763 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:56.095850 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.095824 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:20:57.095802784 +0000 UTC m=+2.997681300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:56.196250 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.196226 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:56.196399 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.196275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:56.196442 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.196404 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:56.196480 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.196439 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:20:56.196480 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.196454 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:20:56.196480 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.196460 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:20:57.196441784 +0000 UTC m=+3.098320308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:56.196480 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.196464 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:56.196601 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:56.196498 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:20:57.196487509 +0000 UTC m=+3.098366022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:56.321950 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.321924 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995f0143_41e0_4437_bc4d_4ceaab716052.slice/crio-2efee9f7f9733249a2b4642f39ee6f58c0eaf772c3e4a3fec42869fb6b0ba989 WatchSource:0}: Error finding container 2efee9f7f9733249a2b4642f39ee6f58c0eaf772c3e4a3fec42869fb6b0ba989: Status 404 returned error can't find the container with id 2efee9f7f9733249a2b4642f39ee6f58c0eaf772c3e4a3fec42869fb6b0ba989 Jul 07 19:20:56.371464 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.371440 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6933ad4a_603f_45fc_8821_14edc1cda57e.slice/crio-aca9b929ee1728b5e0892d85757f2331cfbe4a5f891ab6249f973c7434fc5e3a WatchSource:0}: Error finding container aca9b929ee1728b5e0892d85757f2331cfbe4a5f891ab6249f973c7434fc5e3a: Status 404 returned error can't find the container with id aca9b929ee1728b5e0892d85757f2331cfbe4a5f891ab6249f973c7434fc5e3a Jul 07 19:20:56.372762 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.372676 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f7b774_6866_4593_babd_1deaa73c8dd6.slice/crio-d194448457c3a73f09fbf13245d398da34a8caea70eef1f71f0bf8ea644444f3 WatchSource:0}: Error finding container d194448457c3a73f09fbf13245d398da34a8caea70eef1f71f0bf8ea644444f3: Status 404 returned error can't find the container with id d194448457c3a73f09fbf13245d398da34a8caea70eef1f71f0bf8ea644444f3 Jul 07 19:20:56.403080 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.403057 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab8ec61_668d_45b0_99d3_aa501eb38b63.slice/crio-ace1af43f5f565387f855a0bab2b3f8d524ceea32fdedf52eedf5403137cadd1 WatchSource:0}: Error finding container ace1af43f5f565387f855a0bab2b3f8d524ceea32fdedf52eedf5403137cadd1: Status 404 returned error can't find the container with id ace1af43f5f565387f855a0bab2b3f8d524ceea32fdedf52eedf5403137cadd1 Jul 07 19:20:56.411522 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.411503 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be57a06_c44d_4ef3_9e28_a0384c082bdc.slice/crio-d02d7b5076223c575cb46088df9bf343a915359993e46a298b83cd1086a97279 WatchSource:0}: Error finding container d02d7b5076223c575cb46088df9bf343a915359993e46a298b83cd1086a97279: Status 404 returned error can't find the container with id d02d7b5076223c575cb46088df9bf343a915359993e46a298b83cd1086a97279 Jul 07 19:20:56.425095 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:20:56.425073 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e094b07_2084_42a5_9e20_9852d2eea7c5.slice/crio-3c6853261326556a337db9aac6e701bcaa864d61a3fb27c7c0644aa3b1e53329 WatchSource:0}: Error finding container 3c6853261326556a337db9aac6e701bcaa864d61a3fb27c7c0644aa3b1e53329: Status 404 returned error can't find the container with id 3c6853261326556a337db9aac6e701bcaa864d61a3fb27c7c0644aa3b1e53329 Jul 07 19:20:56.544700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.544548 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:15:55 +0000 UTC" deadline="2028-04-08 03:48:22.389543594 +0000 UTC" Jul 07 19:20:56.544700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.544658 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15368h27m25.844891481s" Jul 07 19:20:56.680617 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.680555 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pkr65" event={"ID":"995f0143-41e0-4437-bc4d-4ceaab716052","Type":"ContainerStarted","Data":"2efee9f7f9733249a2b4642f39ee6f58c0eaf772c3e4a3fec42869fb6b0ba989"} Jul 07 19:20:56.682359 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.682330 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" event={"ID":"e1d6e4c9d780b8824beae118385da5e5","Type":"ContainerStarted","Data":"50042c936aa683744708185dea527094eae30d9f08773770242737547f216d19"} Jul 07 19:20:56.684257 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.684235 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b4lgl" event={"ID":"dab8ec61-668d-45b0-99d3-aa501eb38b63","Type":"ContainerStarted","Data":"ace1af43f5f565387f855a0bab2b3f8d524ceea32fdedf52eedf5403137cadd1"} Jul 07 19:20:56.687292 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.687267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" event={"ID":"16f7b774-6866-4593-babd-1deaa73c8dd6","Type":"ContainerStarted","Data":"d194448457c3a73f09fbf13245d398da34a8caea70eef1f71f0bf8ea644444f3"} Jul 07 19:20:56.691557 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.691530 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fm4q2" event={"ID":"ad798b99-e8b2-439e-ba56-c9805dfacd9d","Type":"ContainerStarted","Data":"28b9603f8ee6d90e4a0d820a8da8bf2f402c68cb2156f633d18f1a4813abba1f"} Jul 07 19:20:56.693327 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.693302 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerStarted","Data":"6a2e8709b3f96a7021faf7caa5c7300557fb557c1e29cb66feaa8d9571fe01f0"} Jul 07 19:20:56.699590 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.699567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4w272" event={"ID":"0964a1be-fa18-4329-a4c6-e93935334f52","Type":"ContainerStarted","Data":"4988cc2b182db265670960016378df7274ae8aab3f3201f5af073fdad9244038"} Jul 07 19:20:56.701642 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.701618 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" event={"ID":"4d37125de82a4e9d646d5bc6cbdf2098","Type":"ContainerStarted","Data":"bad73e2fa1f58d7e917792b647924dee598c80cdba758d7a07f23504bca1bbf2"} Jul 07 19:20:56.703049 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.703028 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" event={"ID":"5e094b07-2084-42a5-9e20-9852d2eea7c5","Type":"ContainerStarted","Data":"3c6853261326556a337db9aac6e701bcaa864d61a3fb27c7c0644aa3b1e53329"} Jul 07 19:20:56.704623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.704602 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"d02d7b5076223c575cb46088df9bf343a915359993e46a298b83cd1086a97279"} Jul 07 19:20:56.708669 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.708647 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d7b7z" event={"ID":"6933ad4a-603f-45fc-8821-14edc1cda57e","Type":"ContainerStarted","Data":"aca9b929ee1728b5e0892d85757f2331cfbe4a5f891ab6249f973c7434fc5e3a"} Jul 07 19:20:56.789670 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:56.789459 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:20:57.102261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.102230 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:57.102465 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.102369 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:57.102465 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.102442 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:20:59.102424564 +0000 UTC m=+5.004303091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:57.202993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.202953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:57.203167 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.203030 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:57.203167 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.203142 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:57.203282 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.203200 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:20:59.203183581 +0000 UTC m=+5.105062096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:57.203731 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.203606 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:20:57.203731 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.203628 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:20:57.203731 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.203640 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:57.203731 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.203683 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:20:59.20366889 +0000 UTC m=+5.105547406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:57.343491 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.343286 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 19:20:57.545023 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.544982 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:15:55 +0000 UTC" deadline="2028-03-13 04:17:10.815309888 +0000 UTC" Jul 07 19:20:57.545023 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.545020 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14744h56m13.270294195s" Jul 07 19:20:57.617165 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.617135 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:57.617493 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.617476 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:57.617575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:57.617498 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:57.617629 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.617562 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:20:57.617686 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.617653 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:20:57.617738 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:57.617714 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:20:59.120152 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:59.119533 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:59.120152 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.119675 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:59.120152 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.119736 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:03.119717369 +0000 UTC m=+9.021595905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:20:59.220930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:59.220886 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:59.221109 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:59.221000 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:59.221202 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.221175 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:20:59.221275 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.221211 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:20:59.221275 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.221225 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:59.221394 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.221282 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:21:03.221264145 +0000 UTC m=+9.123142665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:20:59.221723 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.221693 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:59.221822 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.221754 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:21:03.221739196 +0000 UTC m=+9.123617713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:20:59.617118 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:59.617060 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:20:59.617118 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:59.617099 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:20:59.617328 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.617184 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:20:59.617328 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:20:59.617241 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:20:59.617602 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.617448 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:20:59.617602 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:20:59.617563 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:01.617582 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:01.617550 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:01.618035 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:01.617563 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:01.618035 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:01.617680 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:01.618035 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:01.617692 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:01.618035 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:01.617793 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:01.618035 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:01.617875 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:03.154822 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:03.154784 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:03.155297 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.154981 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:03.155297 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.155046 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:11.155028205 +0000 UTC m=+17.056906720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:03.255571 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:03.255541 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:03.255757 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:03.255588 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:03.255757 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.255712 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:03.255757 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.255744 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:03.255906 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.255768 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:03.255906 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.255775 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:21:11.255757455 +0000 UTC m=+17.157635974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:03.255906 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.255781 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:03.255906 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.255828 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:21:11.255812679 +0000 UTC m=+17.157691215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:03.617238 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:03.617207 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:03.617238 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:03.617241 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:03.617510 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:03.617213 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:03.617510 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.617373 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:03.617510 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.617390 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:03.617510 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:03.617449 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:05.617799 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:05.617758 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:05.618264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:05.617817 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:05.618264 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:05.617906 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:05.618264 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:05.618001 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:05.618264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:05.618100 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:05.618264 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:05.618198 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:07.617543 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.617279 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:07.617988 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.617310 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:07.617988 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:07.617607 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:07.617988 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.617346 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:07.617988 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:07.617689 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:07.617988 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:07.617751 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:07.748244 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.748215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" event={"ID":"e1d6e4c9d780b8824beae118385da5e5","Type":"ContainerStarted","Data":"dcb84435b1ab2f2dfcadb4a770be3880af0e3b905f9de0de4db7ed15e2d6b62d"} Jul 07 19:21:07.749587 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.749556 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" event={"ID":"16f7b774-6866-4593-babd-1deaa73c8dd6","Type":"ContainerStarted","Data":"fdeccd19873f002fc5ded3dc6003fd6ca4cbdbb1477e77ff5cc0a38e644b14c6"} Jul 07 19:21:07.750827 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.750807 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fm4q2" event={"ID":"ad798b99-e8b2-439e-ba56-c9805dfacd9d","Type":"ContainerStarted","Data":"6976c3b343d2259e6ddab016be77c939649504abbda9396e06388da0d17b2c15"} Jul 07 19:21:07.752213 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.752184 2561 generic.go:356] "Generic (PLEG): container finished" podID="338da55c-a41b-4808-8c26-859f730f4ed5" containerID="0898689f7ac5ed9052ed9ecbf5e3ade277333c2078639ccef2dd4dfc8146f087" exitCode=0 Jul 07 19:21:07.752284 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.752213 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerDied","Data":"0898689f7ac5ed9052ed9ecbf5e3ade277333c2078639ccef2dd4dfc8146f087"} Jul 07 19:21:07.753593 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.753569 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4w272" event={"ID":"0964a1be-fa18-4329-a4c6-e93935334f52","Type":"ContainerStarted","Data":"694f028fbc6a3bef8e69d74ac9287c48b8597e1a4fd0c344c20fd653c2dc66c8"} Jul 07 19:21:07.755119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.755070 2561 generic.go:356] "Generic (PLEG): container finished" podID="4d37125de82a4e9d646d5bc6cbdf2098" containerID="6ca6a62a91334da3ea73050892b2e800eebf4c7f20eb07baf8d721c22d07e0fd" exitCode=0 Jul 07 19:21:07.755174 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.755157 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" event={"ID":"4d37125de82a4e9d646d5bc6cbdf2098","Type":"ContainerDied","Data":"6ca6a62a91334da3ea73050892b2e800eebf4c7f20eb07baf8d721c22d07e0fd"} Jul 07 19:21:07.756691 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.756645 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" event={"ID":"5e094b07-2084-42a5-9e20-9852d2eea7c5","Type":"ContainerStarted","Data":"84e13e8b24ca09b07919b2fd450dbde5311ada4441dc48b596b2dbc4d60ba460"} Jul 07 19:21:07.758145 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.758128 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d7b7z" event={"ID":"6933ad4a-603f-45fc-8821-14edc1cda57e","Type":"ContainerStarted","Data":"8cdea0093ac54ccf268eeb44aef05b306cd5a9d68f001c166a067db20f317ed4"} Jul 07 19:21:07.764416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.764341 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" podStartSLOduration=12.764326540999999 podStartE2EDuration="12.764326541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:07.763921661 +0000 UTC m=+13.665800216" watchObservedRunningTime="2026-07-07 19:21:07.764326541 +0000 UTC m=+13.666205078" Jul 07 19:21:07.803718 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.803677 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lqcxh" podStartSLOduration=3.30300133 podStartE2EDuration="13.803664608s" totalImagesPullingTime="10.500663278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:07.803525859 +0000 UTC m=+13.705404404" watchObservedRunningTime="2026-07-07 19:21:07.803664608 +0000 UTC m=+13.705543120" Jul 07 19:21:07.820735 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.820667 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fm4q2" podStartSLOduration=1.979836291 podStartE2EDuration="12.820650089s" totalImagesPullingTime="10.840813798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:07.820582367 +0000 UTC m=+13.722460902" watchObservedRunningTime="2026-07-07 19:21:07.820650089 +0000 UTC m=+13.722528624" Jul 07 19:21:07.871978 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.871463 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d7b7z" podStartSLOduration=3.340721865 podStartE2EDuration="13.871445921s" totalImagesPullingTime="10.530724056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:07.849603701 +0000 UTC m=+13.751482237" watchObservedRunningTime="2026-07-07 19:21:07.871445921 +0000 UTC m=+13.773324455" Jul 07 19:21:07.871978 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:07.871576 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4w272" podStartSLOduration=3.024072959 podStartE2EDuration="13.871565852s" totalImagesPullingTime="10.847492893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:07.871554151 +0000 UTC m=+13.773432710" watchObservedRunningTime="2026-07-07 19:21:07.871565852 +0000 UTC m=+13.773444387" Jul 07 19:21:08.384097 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:08.383862 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:21:08.384632 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:08.384609 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:21:08.761974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:08.761939 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" event={"ID":"4d37125de82a4e9d646d5bc6cbdf2098","Type":"ContainerStarted","Data":"5676b9015ccd77f2a29e6a4613629f5bffe838c7fb740bd682a3c234abe36e48"} Jul 07 19:21:08.763709 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:08.763680 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b4lgl" event={"ID":"dab8ec61-668d-45b0-99d3-aa501eb38b63","Type":"ContainerStarted","Data":"0931774299aaddee594691586ee3ae17651469227a0732fbf45840351e1e1ae1"} Jul 07 19:21:08.781460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:08.781414 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" podStartSLOduration=13.781398891 podStartE2EDuration="13.781398891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:08.780853762 +0000 UTC m=+14.682732298" watchObservedRunningTime="2026-07-07 19:21:08.781398891 +0000 UTC m=+14.683277426" Jul 07 19:21:08.802241 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:08.802199 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b4lgl" podStartSLOduration=4.299625296 podStartE2EDuration="14.802184589s" totalImagesPullingTime="10.502559293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:08.801343453 +0000 UTC m=+14.703221988" watchObservedRunningTime="2026-07-07 19:21:08.802184589 +0000 UTC m=+14.704063124" Jul 07 19:21:09.617039 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:09.617005 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:09.617220 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:09.617103 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:09.617220 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:09.617113 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:09.617220 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:09.617106 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:09.617395 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:09.617207 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:09.617395 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:09.617283 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:09.765659 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:09.765632 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 19:21:11.226338 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:11.226304 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:11.226797 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.226485 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:11.226797 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.226544 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:27.226530621 +0000 UTC m=+33.128409134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:11.327596 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:11.327562 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:11.327768 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:11.327627 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:11.327768 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.327719 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:11.327768 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.327763 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:11.327925 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.327782 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:11.327925 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.327787 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:21:27.327769642 +0000 UTC m=+33.229648154 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:11.327925 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.327797 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:11.327925 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.327848 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:21:27.327831506 +0000 UTC m=+33.229710033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:11.616958 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:11.616881 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:11.617121 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:11.616886 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:11.617121 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.617023 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:11.617121 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:11.616886 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:11.617121 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.617086 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:11.617322 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:11.617151 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:13.617176 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:13.617125 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:13.617176 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:13.617178 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:13.617823 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:13.617237 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:13.617823 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:13.617254 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:13.617823 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:13.617316 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:13.617823 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:13.617421 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:13.853711 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:13.853677 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:21:13.853890 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:13.853824 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 19:21:13.854298 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:13.854266 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d7b7z" Jul 07 19:21:15.616927 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:15.616887 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:15.617358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:15.616909 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:15.617358 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:15.617006 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:15.617358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:15.617066 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:15.617358 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:15.617190 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:15.617358 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:15.617276 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:17.617184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:17.617153 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:17.617798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:17.617189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:17.617798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:17.617189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:17.617798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:17.617254 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:17.617798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:17.617374 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:17.617798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:17.617471 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:19.023845 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.023664 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 19:21:19.569865 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.569779 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T19:21:19.023841997Z","UUID":"43f853b9-e265-4d15-8cb6-7f2064ea7bab","Handler":null,"Name":"","Endpoint":""} Jul 07 19:21:19.571314 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.571296 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 19:21:19.571442 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.571327 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 19:21:19.617231 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.617204 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:19.617409 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.617243 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:19.617409 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:19.617319 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:19.617538 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:19.617417 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:19.617538 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.617486 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:19.617644 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:19.617565 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:19.787443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.787212 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" event={"ID":"16f7b774-6866-4593-babd-1deaa73c8dd6","Type":"ContainerStarted","Data":"730a39f53be4589651b87d5169fb21a282dc316160f7da063235db3eee3047b0"} Jul 07 19:21:19.788805 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.788783 2561 generic.go:356] "Generic (PLEG): container finished" podID="338da55c-a41b-4808-8c26-859f730f4ed5" containerID="9ec3dfc85a2ebe904d1cc07254dbec949dc2129b093c6dcab058ac5c4912d912" exitCode=0 Jul 07 19:21:19.788913 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.788854 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerDied","Data":"9ec3dfc85a2ebe904d1cc07254dbec949dc2129b093c6dcab058ac5c4912d912"} Jul 07 19:21:19.791990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.791901 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"934864d21c7bd941c226bace9ac069d7c0fc532ba294d74a8f1a7b30d112541a"} Jul 07 19:21:19.791990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.791951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"01ea0245c81ec4ec662b5d258e61654cfe2941589da3573563b4f660528b5393"} Jul 07 19:21:19.792100 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.791993 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"7fe97ccd790d9c78b2ab6fd78656a12c9a2e1ed7aafbb1b276be750451616fa2"} Jul 07 19:21:19.792100 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.792040 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"13661b25dc9f54bbe3e5e324759e66c07669f0c29644df19a4f53af6c74fc190"} Jul 07 19:21:19.792100 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.792090 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"22e605c4ebe76842c131c4c1e6d10b199da8328c48231c2428dde35c7d0290d3"} Jul 07 19:21:19.792217 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.792120 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"d0d4648f69538894b076b65719c3b5bfd774989ce4345b0e77c176b6191a15cc"} Jul 07 19:21:19.793467 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.793446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pkr65" event={"ID":"995f0143-41e0-4437-bc4d-4ceaab716052","Type":"ContainerStarted","Data":"1f9541d35e851daa1103b309c515d4897290b63f281fb63e2a9fa50ad49a553e"} Jul 07 19:21:19.834307 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:19.834268 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pkr65" podStartSLOduration=3.271706882 podStartE2EDuration="25.834255772s" totalImagesPullingTime="22.56254889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:19.830088299 +0000 UTC m=+25.731966833" watchObservedRunningTime="2026-07-07 19:21:19.834255772 +0000 UTC m=+25.736134307" Jul 07 19:21:20.796778 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:20.796738 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" event={"ID":"16f7b774-6866-4593-babd-1deaa73c8dd6","Type":"ContainerStarted","Data":"fae6683c40fe5fdbcd9b339f6c8eb8a55c75813e1592f2f13c90bc000bc2fb70"} Jul 07 19:21:20.820990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:20.820942 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kbxvt" podStartSLOduration=3.69388513 podStartE2EDuration="26.820927885s" totalImagesPullingTime="23.127042755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:20.820926804 +0000 UTC m=+26.722805339" watchObservedRunningTime="2026-07-07 19:21:20.820927885 +0000 UTC m=+26.722806420" Jul 07 19:21:21.616976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:21.616953 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:21.617119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:21.616963 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:21.617119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:21.617059 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:21.617119 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:21.617062 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:21.617282 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:21.617122 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:21.617282 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:21.617223 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:21.800683 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:21.800648 2561 generic.go:356] "Generic (PLEG): container finished" podID="338da55c-a41b-4808-8c26-859f730f4ed5" containerID="b88c54ee8ee922b77e97efdf4fb30eeb7ea8ab2a69d8a08e21d6760eff3793c2" exitCode=0 Jul 07 19:21:21.801157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:21.800730 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerDied","Data":"b88c54ee8ee922b77e97efdf4fb30eeb7ea8ab2a69d8a08e21d6760eff3793c2"} Jul 07 19:21:21.804293 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:21.804268 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"0226c0d697c610271b0b6ca92b480a1022a22e9025ad80c1e47f0b07a0ee49f5"} Jul 07 19:21:23.616877 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:23.616842 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:23.616877 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:23.616842 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:23.617465 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:23.616866 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:23.617465 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:23.616975 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:23.617465 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:23.617035 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:23.617465 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:23.617129 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:23.811038 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:23.811015 2561 generic.go:356] "Generic (PLEG): container finished" podID="338da55c-a41b-4808-8c26-859f730f4ed5" containerID="7b78c22812546257c6a00519e51d9823d8547aeaadb566e7061ceb58cee31161" exitCode=0 Jul 07 19:21:23.811151 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:23.811059 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerDied","Data":"7b78c22812546257c6a00519e51d9823d8547aeaadb566e7061ceb58cee31161"} Jul 07 19:21:24.817534 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:24.817137 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" event={"ID":"2be57a06-c44d-4ef3-9e28-a0384c082bdc","Type":"ContainerStarted","Data":"68f2c2fcf684ea37be00e8af22b59df0ac137ae396d57a677909e008364bb6b9"} Jul 07 19:21:24.817534 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:24.817432 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:21:24.817534 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:24.817485 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:21:24.835200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:24.835174 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:21:24.845433 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:24.845352 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" podStartSLOduration=8.361153032 podStartE2EDuration="30.845333587s" totalImagesPullingTime="22.484180555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:24.844824814 +0000 UTC m=+30.746703365" watchObservedRunningTime="2026-07-07 19:21:24.845333587 +0000 UTC m=+30.747212122" Jul 07 19:21:25.617498 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:25.617468 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:25.617658 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:25.617469 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:25.617658 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:25.617630 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:25.617658 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:25.617468 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:25.617780 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:25.617694 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:25.617780 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:25.617752 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:25.820912 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:25.820850 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:21:25.839704 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:25.839676 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:21:26.086858 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:26.086807 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j474f"] Jul 07 19:21:26.087065 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:26.086932 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:26.087146 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:26.087056 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:26.087884 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:26.087675 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-88vw9"] Jul 07 19:21:26.087884 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:26.087766 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:26.088075 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:26.087896 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:26.088458 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:26.088432 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cvqcz"] Jul 07 19:21:26.088577 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:26.088525 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:26.088635 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:26.088620 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:27.247301 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:27.247271 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:27.247794 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.247457 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:27.247794 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.247541 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs podName:92f94e51-0ce7-4298-95ea-b6df5e03d944 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:59.247517484 +0000 UTC m=+65.149396003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs") pod "network-metrics-daemon-cvqcz" (UID: "92f94e51-0ce7-4298-95ea-b6df5e03d944") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 19:21:27.347784 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:27.347749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:27.347963 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:27.347819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:27.347963 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.347916 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:27.347963 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.347933 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 19:21:27.347963 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.347951 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 19:21:27.347963 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.347963 2561 projected.go:196] Error preparing data for projected volume kube-api-access-pvb2q for pod openshift-network-diagnostics/network-check-target-88vw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:27.348201 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.348002 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret podName:b23202de-78de-407f-b354-2c83ecd07eed nodeName:}" failed. No retries permitted until 2026-07-07 19:21:59.34797514 +0000 UTC m=+65.249853653 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret") pod "global-pull-secret-syncer-j474f" (UID: "b23202de-78de-407f-b354-2c83ecd07eed") : object "kube-system"/"original-pull-secret" not registered Jul 07 19:21:27.348201 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.348023 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q podName:fd4b129d-5629-40e3-b586-3e89d67d703b nodeName:}" failed. No retries permitted until 2026-07-07 19:21:59.348013083 +0000 UTC m=+65.249891598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pvb2q" (UniqueName: "kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q") pod "network-check-target-88vw9" (UID: "fd4b129d-5629-40e3-b586-3e89d67d703b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 19:21:27.617052 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:27.616963 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:27.617209 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:27.617076 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:27.617209 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.617083 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:27.617209 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:27.617084 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:27.617209 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.617183 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:27.617427 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:27.617247 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:29.617679 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:29.617489 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:29.618119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:29.617533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:29.618119 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:29.617747 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88vw9" podUID="fd4b129d-5629-40e3-b586-3e89d67d703b" Jul 07 19:21:29.618119 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:29.617865 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvqcz" podUID="92f94e51-0ce7-4298-95ea-b6df5e03d944" Jul 07 19:21:29.618119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:29.617539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:29.618119 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:29.617965 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j474f" podUID="b23202de-78de-407f-b354-2c83ecd07eed" Jul 07 19:21:29.831898 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:29.831861 2561 generic.go:356] "Generic (PLEG): container finished" podID="338da55c-a41b-4808-8c26-859f730f4ed5" containerID="bb9132e762b67a71ec46cb0cd36ca5dafadd2fc8a1c7e21323bcd0383fd92790" exitCode=0 Jul 07 19:21:29.832053 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:29.831924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerDied","Data":"bb9132e762b67a71ec46cb0cd36ca5dafadd2fc8a1c7e21323bcd0383fd92790"} Jul 07 19:21:30.836951 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:30.836917 2561 generic.go:356] "Generic (PLEG): container finished" podID="338da55c-a41b-4808-8c26-859f730f4ed5" containerID="f9634ff9daba274eda7c87045a641de8a6f3c716ddc6b13b905700756125dd11" exitCode=0 Jul 07 19:21:30.837336 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:30.836963 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerDied","Data":"f9634ff9daba274eda7c87045a641de8a6f3c716ddc6b13b905700756125dd11"} Jul 07 19:21:31.375861 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.375837 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeReady" Jul 07 19:21:31.375976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.375966 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 19:21:31.410856 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.410823 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78db467dbd-64sfj"] Jul 07 19:21:31.427806 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.427783 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c"] Jul 07 19:21:31.427908 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.427893 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.435789 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.435771 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7c47h"] Jul 07 19:21:31.435910 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.435896 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.445902 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.445883 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6f4d8db47c-k2k2k"] Jul 07 19:21:31.446019 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.446002 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.452639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.452622 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 19:21:31.453257 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.453235 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 19:21:31.458068 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.458051 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6"] Jul 07 19:21:31.458164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.458149 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.469608 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.469592 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 19:21:31.470114 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.470101 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 19:21:31.472424 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.472409 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2"] Jul 07 19:21:31.472518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.472505 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:31.472950 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.472931 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 19:21:31.473077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.472931 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jrgxk\"" Jul 07 19:21:31.473077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.472935 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6t2z8\"" Jul 07 19:21:31.482665 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.482649 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tbdsk"] Jul 07 19:21:31.482758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.482745 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" Jul 07 19:21:31.490299 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.490282 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78db467dbd-64sfj"] Jul 07 19:21:31.490393 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.490309 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7c47h"] Jul 07 19:21:31.490393 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.490326 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6"] Jul 07 19:21:31.490393 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.490357 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v8bmn"] Jul 07 19:21:31.490492 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.490410 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.497441 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.497421 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.497532 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.497489 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 19:21:31.500309 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.500294 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8"] Jul 07 19:21:31.500434 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.500420 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.501737 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.501723 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 19:21:31.508645 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.508629 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 19:21:31.511957 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.511936 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.514431 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.514413 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml"] Jul 07 19:21:31.514554 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.514539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.517164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.516949 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9s9nl\"" Jul 07 19:21:31.518134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.517310 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.518211 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.518189 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.527809 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.527790 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 19:21:31.527901 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.527819 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.527901 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.527879 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.528098 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.528083 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 19:21:31.528169 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.528097 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 19:21:31.528219 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.528188 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cphpf\"" Jul 07 19:21:31.532190 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532176 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 19:21:31.532404 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532367 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.532450 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532400 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.532642 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532620 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.532642 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532630 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.532795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532666 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.532795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532685 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 19:21:31.532795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532709 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 19:21:31.532795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.532625 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.538621 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.538606 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9"] Jul 07 19:21:31.538732 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.538719 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" Jul 07 19:21:31.545954 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.545937 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lxjvf\"" Jul 07 19:21:31.552070 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.552055 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 19:21:31.552458 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.552444 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2nvbg\"" Jul 07 19:21:31.552584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.552571 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 19:21:31.552885 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.552870 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9nn6n\"" Jul 07 19:21:31.553010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.552996 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wlnmg\"" Jul 07 19:21:31.553141 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.553129 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mchsk\"" Jul 07 19:21:31.558595 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.558573 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.558746 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.558733 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 19:21:31.558781 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.558747 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bdbsg"] Jul 07 19:21:31.558859 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.558848 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.573622 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.573576 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.574951 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.574928 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6f4d8db47c-k2k2k"] Jul 07 19:21:31.575010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.574971 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tbdsk"] Jul 07 19:21:31.575010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.574986 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c"] Jul 07 19:21:31.575010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.574999 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9"] Jul 07 19:21:31.575097 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.574931 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.575126 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.575011 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8"] Jul 07 19:21:31.575154 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.575146 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2"] Jul 07 19:21:31.577771 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.577751 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdbsg"] Jul 07 19:21:31.580687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580663 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb275177-b68f-440a-b440-9fa8d5f4c2c3-serving-cert\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580743 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c36715ae-d4d5-4567-b01d-7997b87573ac-ca-trust-extracted\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580797 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbvvt\" (UniqueName: \"kubernetes.io/projected/5088af8e-6f05-456a-a676-6121c47d54b6-kube-api-access-tbvvt\") pod \"volume-data-source-validator-5d456fdbcd-r85f2\" (UID: \"5088af8e-6f05-456a-a676-6121c47d54b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-config\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580884 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207323e-9118-4aba-bf0d-47b77fd13358-tmp-dir\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb275177-b68f-440a-b440-9fa8d5f4c2c3-tmp\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580939 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9207323e-9118-4aba-bf0d-47b77fd13358-config-volume\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.580991 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xvrp\" (UniqueName: \"kubernetes.io/projected/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-kube-api-access-6xvrp\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.581013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581016 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-installation-pull-secrets\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-default-certificate\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7plnh\" (UniqueName: \"kubernetes.io/projected/00bb5516-a12a-4606-b281-d2cb60b6a0f8-kube-api-access-7plnh\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcphr\" (UniqueName: \"kubernetes.io/projected/9207323e-9118-4aba-bf0d-47b77fd13358-kube-api-access-rcphr\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581114 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-image-registry-private-configuration\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581141 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxwsh\" (UniqueName: \"kubernetes.io/projected/fb275177-b68f-440a-b440-9fa8d5f4c2c3-kube-api-access-sxwsh\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-trusted-ca\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581240 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb275177-b68f-440a-b440-9fa8d5f4c2c3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581276 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581307 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-serving-cert\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581352 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581431 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxtrz\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-kube-api-access-cxtrz\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4wvb\" (UniqueName: \"kubernetes.io/projected/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-kube-api-access-p4wvb\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581512 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.581611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-trusted-ca\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-bound-sa-token\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581620 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggtr\" (UniqueName: \"kubernetes.io/projected/f22b39b2-3315-48e7-b2a4-70ead9b74898-kube-api-access-fggtr\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581690 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-certificates\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-stats-auth\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb275177-b68f-440a-b440-9fa8d5f4c2c3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.582266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.581863 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fb275177-b68f-440a-b440-9fa8d5f4c2c3-snapshots\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.583727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.583710 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.583899 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.583881 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 19:21:31.584034 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584018 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hvs4x\"" Jul 07 19:21:31.584095 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584042 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 19:21:31.584364 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584341 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-99hqg\"" Jul 07 19:21:31.584462 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584365 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 19:21:31.584462 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584442 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.584613 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584578 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.584843 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584707 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:31.584843 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584771 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.585051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584931 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 19:21:31.585051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584963 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dwrjw\"" Jul 07 19:21:31.585051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.584985 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8bmn"] Jul 07 19:21:31.585203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.585124 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 19:21:31.585615 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.585601 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml"] Jul 07 19:21:31.617019 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.616999 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:31.617115 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.617044 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:31.617115 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.617114 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:31.631702 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.631683 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 19:21:31.636550 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.636533 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 19:21:31.636633 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.636579 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 19:21:31.636777 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.636762 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rq87d\"" Jul 07 19:21:31.636872 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.636857 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhh8r\"" Jul 07 19:21:31.638604 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.638589 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 19:21:31.641677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.641651 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 19:21:31.682182 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-image-registry-private-configuration\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.682264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682188 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxwsh\" (UniqueName: \"kubernetes.io/projected/fb275177-b68f-440a-b440-9fa8d5f4c2c3-kube-api-access-sxwsh\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.682264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-trusted-ca\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.682264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682228 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb275177-b68f-440a-b440-9fa8d5f4c2c3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.682351 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.682412 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682348 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b6f199d-ac3f-445b-8458-bbba5009bd0c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.682412 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.682495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-serving-cert\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.682495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.682594 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxtrz\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-kube-api-access-cxtrz\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.682646 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4wvb\" (UniqueName: \"kubernetes.io/projected/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-kube-api-access-p4wvb\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.682646 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.682629 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 19:21:31.682646 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682639 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.682773 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.682660 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 19:21:31.682773 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682668 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgqzz\" (UniqueName: \"kubernetes.io/projected/6b6f199d-ac3f-445b-8458-bbba5009bd0c-kube-api-access-kgqzz\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.682773 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.682703 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls podName:9207323e-9118-4aba-bf0d-47b77fd13358 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:32.182681698 +0000 UTC m=+38.084560233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls") pod "dns-default-v8bmn" (UID: "9207323e-9118-4aba-bf0d-47b77fd13358") : secret "dns-default-metrics-tls" not found Jul 07 19:21:31.682919 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.682846 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb275177-b68f-440a-b440-9fa8d5f4c2c3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.683024 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.682984 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:32.182966776 +0000 UTC m=+38.084845294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : secret "router-metrics-certs-default" not found Jul 07 19:21:31.683024 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.683024 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:32.183013844 +0000 UTC m=+38.084892362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : configmap references non-existent config key: service-ca.crt Jul 07 19:21:31.683179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683047 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-trusted-ca\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.683179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-bound-sa-token\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.683179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:31.683179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683141 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.683179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683150 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggtr\" (UniqueName: \"kubernetes.io/projected/f22b39b2-3315-48e7-b2a4-70ead9b74898-kube-api-access-fggtr\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-certificates\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-stats-auth\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb275177-b68f-440a-b440-9fa8d5f4c2c3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.683240 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f867d34-c4ae-44fb-a644-3faa3118dc5d-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.683299 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls podName:f22b39b2-3315-48e7-b2a4-70ead9b74898 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:32.183283862 +0000 UTC m=+38.085162375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-gzfc6" (UID: "f22b39b2-3315-48e7-b2a4-70ead9b74898") : secret "samples-operator-tls" not found Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683308 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-trusted-ca\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683319 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b6f199d-ac3f-445b-8458-bbba5009bd0c-config\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ecf5971-da77-445e-b564-b403dcdae2ee-cert\") pod \"ingress-canary-bdbsg\" (UID: \"3ecf5971-da77-445e-b564-b403dcdae2ee\") " pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fb275177-b68f-440a-b440-9fa8d5f4c2c3-snapshots\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.683447 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683439 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6wb\" (UniqueName: \"kubernetes.io/projected/8f867d34-c4ae-44fb-a644-3faa3118dc5d-kube-api-access-4x6wb\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683482 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f867d34-c4ae-44fb-a644-3faa3118dc5d-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683509 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffgk\" (UniqueName: \"kubernetes.io/projected/ddc20fe2-be24-4780-8102-5099a044422d-kube-api-access-rffgk\") pod \"network-check-source-989f8b8dc-l78ml\" (UID: \"ddc20fe2-be24-4780-8102-5099a044422d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683541 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb275177-b68f-440a-b440-9fa8d5f4c2c3-serving-cert\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683618 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683645 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c36715ae-d4d5-4567-b01d-7997b87573ac-ca-trust-extracted\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683668 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbvvt\" (UniqueName: \"kubernetes.io/projected/5088af8e-6f05-456a-a676-6121c47d54b6-kube-api-access-tbvvt\") pod \"volume-data-source-validator-5d456fdbcd-r85f2\" (UID: \"5088af8e-6f05-456a-a676-6121c47d54b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683691 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-config\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207323e-9118-4aba-bf0d-47b77fd13358-tmp-dir\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683748 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9xp4\" (UniqueName: \"kubernetes.io/projected/3ecf5971-da77-445e-b564-b403dcdae2ee-kube-api-access-h9xp4\") pod \"ingress-canary-bdbsg\" (UID: \"3ecf5971-da77-445e-b564-b403dcdae2ee\") " pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683777 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb275177-b68f-440a-b440-9fa8d5f4c2c3-tmp\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9207323e-9118-4aba-bf0d-47b77fd13358-config-volume\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683848 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xvrp\" (UniqueName: \"kubernetes.io/projected/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-kube-api-access-6xvrp\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683862 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-trusted-ca\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683882 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-installation-pull-secrets\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.684007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-default-certificate\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683937 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7plnh\" (UniqueName: \"kubernetes.io/projected/00bb5516-a12a-4606-b281-d2cb60b6a0f8-kube-api-access-7plnh\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.683944 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.683966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcphr\" (UniqueName: \"kubernetes.io/projected/9207323e-9118-4aba-bf0d-47b77fd13358-kube-api-access-rcphr\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.683999 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls podName:4983fdb8-9403-40d2-bc8e-18ecd0ee6b74 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:32.183984225 +0000 UTC m=+38.085862753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-26j4c" (UID: "4983fdb8-9403-40d2-bc8e-18ecd0ee6b74") : secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.684012 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-certificates\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.684270 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.684284 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78db467dbd-64sfj: secret "image-registry-tls" not found Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.684289 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c36715ae-d4d5-4567-b01d-7997b87573ac-ca-trust-extracted\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.684298 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb275177-b68f-440a-b440-9fa8d5f4c2c3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.684563 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fb275177-b68f-440a-b440-9fa8d5f4c2c3-snapshots\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:31.684924 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls podName:c36715ae-d4d5-4567-b01d-7997b87573ac nodeName:}" failed. No retries permitted until 2026-07-07 19:21:32.184906514 +0000 UTC m=+38.086785030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls") pod "image-registry-78db467dbd-64sfj" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac") : secret "image-registry-tls" not found Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.684943 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207323e-9118-4aba-bf0d-47b77fd13358-tmp-dir\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.685060 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-config\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.685189 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb275177-b68f-440a-b440-9fa8d5f4c2c3-tmp\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.685700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.685470 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9207323e-9118-4aba-bf0d-47b77fd13358-config-volume\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.687270 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.687248 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-serving-cert\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.687354 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.687272 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-installation-pull-secrets\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.687354 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.687335 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-image-registry-private-configuration\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.687496 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.687441 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-stats-auth\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.687713 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.687692 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-default-certificate\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.687847 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.687833 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb275177-b68f-440a-b440-9fa8d5f4c2c3-serving-cert\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.701630 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.701611 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxwsh\" (UniqueName: \"kubernetes.io/projected/fb275177-b68f-440a-b440-9fa8d5f4c2c3-kube-api-access-sxwsh\") pod \"insights-operator-5c847dcfc4-7c47h\" (UID: \"fb275177-b68f-440a-b440-9fa8d5f4c2c3\") " pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.703706 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.703681 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggtr\" (UniqueName: \"kubernetes.io/projected/f22b39b2-3315-48e7-b2a4-70ead9b74898-kube-api-access-fggtr\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:31.704817 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.704789 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xvrp\" (UniqueName: \"kubernetes.io/projected/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-kube-api-access-6xvrp\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:31.704921 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.704905 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbvvt\" (UniqueName: \"kubernetes.io/projected/5088af8e-6f05-456a-a676-6121c47d54b6-kube-api-access-tbvvt\") pod \"volume-data-source-validator-5d456fdbcd-r85f2\" (UID: \"5088af8e-6f05-456a-a676-6121c47d54b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" Jul 07 19:21:31.705020 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.705005 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4wvb\" (UniqueName: \"kubernetes.io/projected/ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5-kube-api-access-p4wvb\") pod \"console-operator-68b757fb67-tbdsk\" (UID: \"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5\") " pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.706350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.706315 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcphr\" (UniqueName: \"kubernetes.io/projected/9207323e-9118-4aba-bf0d-47b77fd13358-kube-api-access-rcphr\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:31.708585 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.708566 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-bound-sa-token\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.711806 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.711789 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7plnh\" (UniqueName: \"kubernetes.io/projected/00bb5516-a12a-4606-b281-d2cb60b6a0f8-kube-api-access-7plnh\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:31.718888 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.718870 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxtrz\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-kube-api-access-cxtrz\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:31.754147 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.754122 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" Jul 07 19:21:31.786117 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786096 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b6f199d-ac3f-445b-8458-bbba5009bd0c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.786229 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786173 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgqzz\" (UniqueName: \"kubernetes.io/projected/6b6f199d-ac3f-445b-8458-bbba5009bd0c-kube-api-access-kgqzz\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.786286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786228 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f867d34-c4ae-44fb-a644-3faa3118dc5d-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.786286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b6f199d-ac3f-445b-8458-bbba5009bd0c-config\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.786393 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ecf5971-da77-445e-b564-b403dcdae2ee-cert\") pod \"ingress-canary-bdbsg\" (UID: \"3ecf5971-da77-445e-b564-b403dcdae2ee\") " pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.786503 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786475 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6wb\" (UniqueName: \"kubernetes.io/projected/8f867d34-c4ae-44fb-a644-3faa3118dc5d-kube-api-access-4x6wb\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.786593 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f867d34-c4ae-44fb-a644-3faa3118dc5d-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.786593 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786571 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffgk\" (UniqueName: \"kubernetes.io/projected/ddc20fe2-be24-4780-8102-5099a044422d-kube-api-access-rffgk\") pod \"network-check-source-989f8b8dc-l78ml\" (UID: \"ddc20fe2-be24-4780-8102-5099a044422d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" Jul 07 19:21:31.786694 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9xp4\" (UniqueName: \"kubernetes.io/projected/3ecf5971-da77-445e-b564-b403dcdae2ee-kube-api-access-h9xp4\") pod \"ingress-canary-bdbsg\" (UID: \"3ecf5971-da77-445e-b564-b403dcdae2ee\") " pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.786918 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.786887 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b6f199d-ac3f-445b-8458-bbba5009bd0c-config\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.787193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.787173 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f867d34-c4ae-44fb-a644-3faa3118dc5d-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.788428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.788401 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b6f199d-ac3f-445b-8458-bbba5009bd0c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.788740 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.788721 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ecf5971-da77-445e-b564-b403dcdae2ee-cert\") pod \"ingress-canary-bdbsg\" (UID: \"3ecf5971-da77-445e-b564-b403dcdae2ee\") " pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.788845 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.788829 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f867d34-c4ae-44fb-a644-3faa3118dc5d-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.790839 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.790821 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" Jul 07 19:21:31.798271 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.798250 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgqzz\" (UniqueName: \"kubernetes.io/projected/6b6f199d-ac3f-445b-8458-bbba5009bd0c-kube-api-access-kgqzz\") pod \"service-ca-operator-5d46d9ffd4-r7tl9\" (UID: \"6b6f199d-ac3f-445b-8458-bbba5009bd0c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.798419 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.798400 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:31.800403 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.800365 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6wb\" (UniqueName: \"kubernetes.io/projected/8f867d34-c4ae-44fb-a644-3faa3118dc5d-kube-api-access-4x6wb\") pod \"kube-storage-version-migrator-operator-846bf668f7-jdkm8\" (UID: \"8f867d34-c4ae-44fb-a644-3faa3118dc5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.801217 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.801193 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9xp4\" (UniqueName: \"kubernetes.io/projected/3ecf5971-da77-445e-b564-b403dcdae2ee-kube-api-access-h9xp4\") pod \"ingress-canary-bdbsg\" (UID: \"3ecf5971-da77-445e-b564-b403dcdae2ee\") " pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.801596 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.801576 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffgk\" (UniqueName: \"kubernetes.io/projected/ddc20fe2-be24-4780-8102-5099a044422d-kube-api-access-rffgk\") pod \"network-check-source-989f8b8dc-l78ml\" (UID: \"ddc20fe2-be24-4780-8102-5099a044422d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" Jul 07 19:21:31.825720 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.825652 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" Jul 07 19:21:31.847054 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.846839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" Jul 07 19:21:31.874779 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.874569 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" Jul 07 19:21:31.888399 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.884621 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdbsg" Jul 07 19:21:31.888399 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.885163 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" event={"ID":"338da55c-a41b-4808-8c26-859f730f4ed5","Type":"ContainerStarted","Data":"ab9cfea3378c587a10e73f921163058826379fce1354b48061130892a01c22e8"} Jul 07 19:21:31.935340 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:31.934621 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4wnkk" podStartSLOduration=8.639321793 podStartE2EDuration="37.934598345s" totalImagesPullingTime="29.295276552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:31.933193971 +0000 UTC m=+37.835072526" watchObservedRunningTime="2026-07-07 19:21:31.934598345 +0000 UTC m=+37.836476881" Jul 07 19:21:32.040297 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.040270 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2"] Jul 07 19:21:32.042947 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.042879 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7c47h"] Jul 07 19:21:32.060338 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.060307 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tbdsk"] Jul 07 19:21:32.061770 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:32.061740 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4bb462_baf6_4e2f_94f8_cb50aa27cbd5.slice/crio-9320e0e3b046429a1d6cfe2654401ea229f2fa5a4796b9e4755aedfd5d27bd88 WatchSource:0}: Error finding container 9320e0e3b046429a1d6cfe2654401ea229f2fa5a4796b9e4755aedfd5d27bd88: Status 404 returned error can't find the container with id 9320e0e3b046429a1d6cfe2654401ea229f2fa5a4796b9e4755aedfd5d27bd88 Jul 07 19:21:32.073404 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.073206 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:32.076113 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.076090 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8"] Jul 07 19:21:32.077973 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:32.077941 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f867d34_c4ae_44fb_a644_3faa3118dc5d.slice/crio-46c2161a39a9a016e2b6c21bc210b245e929598ac587dd21b7285b019e678924 WatchSource:0}: Error finding container 46c2161a39a9a016e2b6c21bc210b245e929598ac587dd21b7285b019e678924: Status 404 returned error can't find the container with id 46c2161a39a9a016e2b6c21bc210b245e929598ac587dd21b7285b019e678924 Jul 07 19:21:32.082030 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.082004 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml"] Jul 07 19:21:32.094522 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.094504 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdbsg"] Jul 07 19:21:32.096408 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:32.096374 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecf5971_da77_445e_b564_b403dcdae2ee.slice/crio-5df319b5e6756c24ac4aac5f9e95959630ad5a268d2160ba4faceccc7a57253e WatchSource:0}: Error finding container 5df319b5e6756c24ac4aac5f9e95959630ad5a268d2160ba4faceccc7a57253e: Status 404 returned error can't find the container with id 5df319b5e6756c24ac4aac5f9e95959630ad5a268d2160ba4faceccc7a57253e Jul 07 19:21:32.101032 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.101008 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9"] Jul 07 19:21:32.103209 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:32.103192 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b6f199d_ac3f_445b_8458_bbba5009bd0c.slice/crio-90b375117a8ec7a2c6957c667da28fbe8ff2b0cf293ac157a91b29f9a4c018e9 WatchSource:0}: Error finding container 90b375117a8ec7a2c6957c667da28fbe8ff2b0cf293ac157a91b29f9a4c018e9: Status 404 returned error can't find the container with id 90b375117a8ec7a2c6957c667da28fbe8ff2b0cf293ac157a91b29f9a4c018e9 Jul 07 19:21:32.164479 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.164427 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:32.189333 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.189309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:32.189469 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.189343 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:32.189469 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.189407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:32.189469 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.189430 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:32.189575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.189477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:32.189575 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189496 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.189475062 +0000 UTC m=+39.091353586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : configmap references non-existent config key: service-ca.crt Jul 07 19:21:32.189575 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189528 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 19:21:32.189575 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189539 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 19:21:32.189575 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189555 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78db467dbd-64sfj: secret "image-registry-tls" not found Jul 07 19:21:32.189575 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189568 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189543 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.189539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189581 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls podName:f22b39b2-3315-48e7-b2a4-70ead9b74898 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.189566209 +0000 UTC m=+39.091444722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-gzfc6" (UID: "f22b39b2-3315-48e7-b2a4-70ead9b74898") : secret "samples-operator-tls" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189599 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189652 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls podName:c36715ae-d4d5-4567-b01d-7997b87573ac nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.189637245 +0000 UTC m=+39.091515772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls") pod "image-registry-78db467dbd-64sfj" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac") : secret "image-registry-tls" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189670 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls podName:4983fdb8-9403-40d2-bc8e-18ecd0ee6b74 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.18966089 +0000 UTC m=+39.091539408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-26j4c" (UID: "4983fdb8-9403-40d2-bc8e-18ecd0ee6b74") : secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189685 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.189677476 +0000 UTC m=+39.091556002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : secret "router-metrics-certs-default" not found Jul 07 19:21:32.189775 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:32.189697 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls podName:9207323e-9118-4aba-bf0d-47b77fd13358 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:33.189690274 +0000 UTC m=+39.091568792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls") pod "dns-default-v8bmn" (UID: "9207323e-9118-4aba-bf0d-47b77fd13358") : secret "dns-default-metrics-tls" not found Jul 07 19:21:32.891223 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.891186 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" event={"ID":"6b6f199d-ac3f-445b-8458-bbba5009bd0c","Type":"ContainerStarted","Data":"90b375117a8ec7a2c6957c667da28fbe8ff2b0cf293ac157a91b29f9a4c018e9"} Jul 07 19:21:32.895369 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.895295 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" event={"ID":"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5","Type":"ContainerStarted","Data":"9320e0e3b046429a1d6cfe2654401ea229f2fa5a4796b9e4755aedfd5d27bd88"} Jul 07 19:21:32.897621 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.897553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" event={"ID":"ddc20fe2-be24-4780-8102-5099a044422d","Type":"ContainerStarted","Data":"1429aec2b90394fe744ab40e32f6553636831731d7a8edc21aef42cfd01e4838"} Jul 07 19:21:32.899515 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.899440 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" event={"ID":"5088af8e-6f05-456a-a676-6121c47d54b6","Type":"ContainerStarted","Data":"66537f8d06f5d924bb8bab34096606f56438ae9f849671afe876523b3eaaef97"} Jul 07 19:21:32.900901 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.900821 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" event={"ID":"fb275177-b68f-440a-b440-9fa8d5f4c2c3","Type":"ContainerStarted","Data":"5f337bf5bcc72f9e0f4d6141bdcb6d71c7a6c4a7c616f228befbb7678e7071b8"} Jul 07 19:21:32.903348 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.903292 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdbsg" event={"ID":"3ecf5971-da77-445e-b564-b403dcdae2ee","Type":"ContainerStarted","Data":"5df319b5e6756c24ac4aac5f9e95959630ad5a268d2160ba4faceccc7a57253e"} Jul 07 19:21:32.907504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:32.907481 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" event={"ID":"8f867d34-c4ae-44fb-a644-3faa3118dc5d","Type":"ContainerStarted","Data":"46c2161a39a9a016e2b6c21bc210b245e929598ac587dd21b7285b019e678924"} Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:33.198959 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:33.199100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:33.199183 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:33.199209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:33.199244 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:33.199277 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199527 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199590 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:35.199572167 +0000 UTC m=+41.101450683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : secret "router-metrics-certs-default" not found Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199684 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 19:21:33.199798 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199746 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls podName:f22b39b2-3315-48e7-b2a4-70ead9b74898 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:35.199731967 +0000 UTC m=+41.101610487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-gzfc6" (UID: "f22b39b2-3315-48e7-b2a4-70ead9b74898") : secret "samples-operator-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199818 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199851 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls podName:9207323e-9118-4aba-bf0d-47b77fd13358 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:35.1998416 +0000 UTC m=+41.101720114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls") pod "dns-default-v8bmn" (UID: "9207323e-9118-4aba-bf0d-47b77fd13358") : secret "dns-default-metrics-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199911 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:35.199902192 +0000 UTC m=+41.101780706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : configmap references non-existent config key: service-ca.crt Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199964 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.199991 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls podName:4983fdb8-9403-40d2-bc8e-18ecd0ee6b74 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:35.199981647 +0000 UTC m=+41.101860162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-26j4c" (UID: "4983fdb8-9403-40d2-bc8e-18ecd0ee6b74") : secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.200048 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.200059 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78db467dbd-64sfj: secret "image-registry-tls" not found Jul 07 19:21:33.200444 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:33.200089 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls podName:c36715ae-d4d5-4567-b01d-7997b87573ac nodeName:}" failed. No retries permitted until 2026-07-07 19:21:35.200079653 +0000 UTC m=+41.101958180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls") pod "image-registry-78db467dbd-64sfj" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac") : secret "image-registry-tls" not found Jul 07 19:21:34.917110 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:34.917070 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" event={"ID":"5088af8e-6f05-456a-a676-6121c47d54b6","Type":"ContainerStarted","Data":"b1fadeb7cebcbc4371e5a3e2c4e52c653583851ce6c9d2dc53b70c38b8b16c6b"} Jul 07 19:21:35.036167 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.036128 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:35.137175 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.137141 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:35.219273 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.219232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:35.219451 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219370 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 19:21:35.219451 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219409 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78db467dbd-64sfj: secret "image-registry-tls" not found Jul 07 19:21:35.219451 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.219407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219466 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls podName:c36715ae-d4d5-4567-b01d-7997b87573ac nodeName:}" failed. No retries permitted until 2026-07-07 19:21:39.219447812 +0000 UTC m=+45.121326337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls") pod "image-registry-78db467dbd-64sfj" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac") : secret "image-registry-tls" not found Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219510 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.219525 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219546 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls podName:4983fdb8-9403-40d2-bc8e-18ecd0ee6b74 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:39.219535852 +0000 UTC m=+45.121414364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-26j4c" (UID: "4983fdb8-9403-40d2-bc8e-18ecd0ee6b74") : secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.219572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219584 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.219614 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:39.219604737 +0000 UTC m=+45.121483262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : secret "router-metrics-certs-default" not found Jul 07 19:21:35.219624 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.219607 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:35.219947 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:35.219649 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:35.220056 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.220035 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 19:21:35.220123 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.220094 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:39.220063555 +0000 UTC m=+45.121942076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : configmap references non-existent config key: service-ca.crt Jul 07 19:21:35.220123 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.220115 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls podName:9207323e-9118-4aba-bf0d-47b77fd13358 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:39.220104683 +0000 UTC m=+45.121983196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls") pod "dns-default-v8bmn" (UID: "9207323e-9118-4aba-bf0d-47b77fd13358") : secret "dns-default-metrics-tls" not found Jul 07 19:21:35.220221 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.220182 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 19:21:35.220268 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:35.220229 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls podName:f22b39b2-3315-48e7-b2a4-70ead9b74898 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:39.220212805 +0000 UTC m=+45.122091333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-gzfc6" (UID: "f22b39b2-3315-48e7-b2a4-70ead9b74898") : secret "samples-operator-tls" not found Jul 07 19:21:36.951857 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:36.951827 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:37.056687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:37.056652 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:38.156213 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:38.156027 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:38.227597 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:38.227546 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:38.877785 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:38.877716 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:39.214857 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.214814 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:39.257840 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.257818 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:39.257916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.257879 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:39.257916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.257910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:39.257990 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.257976 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 19:21:39.258023 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258016 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:39.258061 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258022 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 19:21:39.258061 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258036 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78db467dbd-64sfj: secret "image-registry-tls" not found Jul 07 19:21:39.258061 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258047 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls podName:f22b39b2-3315-48e7-b2a4-70ead9b74898 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:47.258026862 +0000 UTC m=+53.159905378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-gzfc6" (UID: "f22b39b2-3315-48e7-b2a4-70ead9b74898") : secret "samples-operator-tls" not found Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258072 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls podName:c36715ae-d4d5-4567-b01d-7997b87573ac nodeName:}" failed. No retries permitted until 2026-07-07 19:21:47.258060467 +0000 UTC m=+53.159939005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls") pod "image-registry-78db467dbd-64sfj" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac") : secret "image-registry-tls" not found Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.258071 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258088 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls podName:4983fdb8-9403-40d2-bc8e-18ecd0ee6b74 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:47.258078774 +0000 UTC m=+53.159957290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-26j4c" (UID: "4983fdb8-9403-40d2-bc8e-18ecd0ee6b74") : secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.258117 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258130 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.258151 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:39.258184 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258163 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:47.258152919 +0000 UTC m=+53.160031433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : secret "router-metrics-certs-default" not found Jul 07 19:21:39.258431 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258226 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 19:21:39.258431 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258234 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:47.258223821 +0000 UTC m=+53.160102334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : configmap references non-existent config key: service-ca.crt Jul 07 19:21:39.258431 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:39.258265 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls podName:9207323e-9118-4aba-bf0d-47b77fd13358 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:47.258248321 +0000 UTC m=+53.160126850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls") pod "dns-default-v8bmn" (UID: "9207323e-9118-4aba-bf0d-47b77fd13358") : secret "dns-default-metrics-tls" not found Jul 07 19:21:39.328448 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.328415 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:39.942758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.941860 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" event={"ID":"ddc20fe2-be24-4780-8102-5099a044422d","Type":"ContainerStarted","Data":"d2deb2962d00d61dacdc7ea2312f367a2ed329bdaac66dbee65dcfa3adf3d30a"} Jul 07 19:21:39.944671 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.944173 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" event={"ID":"fb275177-b68f-440a-b440-9fa8d5f4c2c3","Type":"ContainerStarted","Data":"ff046c301b22f847995a89e816ad06148db923ffb92a56ba161187490a2c56e3"} Jul 07 19:21:39.946605 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.946562 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdbsg" event={"ID":"3ecf5971-da77-445e-b564-b403dcdae2ee","Type":"ContainerStarted","Data":"f59e984dc128c1225f1719f997258b1d5a8a1fae24375f966cb9e50df52b8ac6"} Jul 07 19:21:39.948976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.948938 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" event={"ID":"8f867d34-c4ae-44fb-a644-3faa3118dc5d","Type":"ContainerStarted","Data":"5bc3965a5f489bd6e824c1fdd43abd7ea536dc1a9db0845b124e24c75ac6d9f0"} Jul 07 19:21:39.950028 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.950006 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" event={"ID":"6b6f199d-ac3f-445b-8458-bbba5009bd0c","Type":"ContainerStarted","Data":"aa83ab0db5b11c1d8840484ca6af4fd234bd3c94936e02fd315f2c045a0f3a7e"} Jul 07 19:21:39.952420 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.952403 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/0.log" Jul 07 19:21:39.952563 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.952546 2561 generic.go:356] "Generic (PLEG): container finished" podID="ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5" containerID="bb17973d1e97ce561bcd96ccc0c3c1b21b1a8d9e1489749850e85e22e702d104" exitCode=255 Jul 07 19:21:39.952661 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.952650 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" event={"ID":"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5","Type":"ContainerDied","Data":"bb17973d1e97ce561bcd96ccc0c3c1b21b1a8d9e1489749850e85e22e702d104"} Jul 07 19:21:39.952971 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.952959 2561 scope.go:117] "RemoveContainer" containerID="bb17973d1e97ce561bcd96ccc0c3c1b21b1a8d9e1489749850e85e22e702d104" Jul 07 19:21:39.966113 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.966069 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r85f2" podStartSLOduration=14.940854304 podStartE2EDuration="16.966052144s" totalImagesPullingTime="2.02519784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:34.937280805 +0000 UTC m=+40.839159362" watchObservedRunningTime="2026-07-07 19:21:39.966052144 +0000 UTC m=+45.867930680" Jul 07 19:21:39.990225 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:39.990180 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-l78ml" podStartSLOduration=2.563764282 podStartE2EDuration="9.990162331s" totalImagesPullingTime="7.426398049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:39.966442811 +0000 UTC m=+45.868321362" watchObservedRunningTime="2026-07-07 19:21:39.990162331 +0000 UTC m=+45.892040867" Jul 07 19:21:40.016049 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.016002 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" podStartSLOduration=4.576613298 podStartE2EDuration="12.015984052s" totalImagesPullingTime="7.439370754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:40.015955665 +0000 UTC m=+45.917834225" watchObservedRunningTime="2026-07-07 19:21:40.015984052 +0000 UTC m=+45.917862588" Jul 07 19:21:40.043077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.043029 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bdbsg" podStartSLOduration=1.627567892 podStartE2EDuration="9.043010734s" totalImagesPullingTime="7.415442842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:40.041665474 +0000 UTC m=+45.943544022" watchObservedRunningTime="2026-07-07 19:21:40.043010734 +0000 UTC m=+45.944889269" Jul 07 19:21:40.062650 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.061252 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" podStartSLOduration=4.652106034 podStartE2EDuration="12.061233372s" totalImagesPullingTime="7.409127338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:40.05966696 +0000 UTC m=+45.961545523" watchObservedRunningTime="2026-07-07 19:21:40.061233372 +0000 UTC m=+45.963111909" Jul 07 19:21:40.737560 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.737527 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:40.959017 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.958993 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:21:40.959573 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.959557 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/0.log" Jul 07 19:21:40.959657 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.959600 2561 generic.go:356] "Generic (PLEG): container finished" podID="ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5" containerID="37fcf83c0fa4d2eb3e0df0d6c79916ecf3a511c5eef1566ee6791a8d18cdee60" exitCode=255 Jul 07 19:21:40.959757 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.959723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" event={"ID":"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5","Type":"ContainerDied","Data":"37fcf83c0fa4d2eb3e0df0d6c79916ecf3a511c5eef1566ee6791a8d18cdee60"} Jul 07 19:21:40.959855 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.959837 2561 scope.go:117] "RemoveContainer" containerID="bb17973d1e97ce561bcd96ccc0c3c1b21b1a8d9e1489749850e85e22e702d104" Jul 07 19:21:40.960866 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.960844 2561 scope.go:117] "RemoveContainer" containerID="37fcf83c0fa4d2eb3e0df0d6c79916ecf3a511c5eef1566ee6791a8d18cdee60" Jul 07 19:21:40.962737 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:40.962041 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tbdsk_openshift-console-operator(ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5)\"" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" podUID="ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5" Jul 07 19:21:40.994353 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:40.994308 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" podStartSLOduration=15.532232465 podStartE2EDuration="22.994287061s" totalImagesPullingTime="7.462054596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:40.094273166 +0000 UTC m=+45.996151704" watchObservedRunningTime="2026-07-07 19:21:40.994287061 +0000 UTC m=+46.896165594" Jul 07 19:21:41.052062 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.052026 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t727j"] Jul 07 19:21:41.088860 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.088833 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.095575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.095550 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 19:21:41.095669 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.095586 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ktbrl\"" Jul 07 19:21:41.095966 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.095943 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 19:21:41.098441 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.098424 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t727j"] Jul 07 19:21:41.135575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.135539 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:41.177797 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.177763 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-data-volume\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.177947 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.177871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.178010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.177980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvvxk\" (UniqueName: \"kubernetes.io/projected/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-api-access-dvvxk\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.178103 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.178085 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.178163 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.178119 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-crio-socket\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.278606 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-data-volume\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.278606 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278589 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.278820 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvvxk\" (UniqueName: \"kubernetes.io/projected/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-api-access-dvvxk\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.278820 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.278820 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-crio-socket\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.278820 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.278720 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 19:21:41.278820 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.278789 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:41.778769831 +0000 UTC m=+47.680648345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : configmap "kube-rbac-proxy" not found Jul 07 19:21:41.278820 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.278822 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 19:21:41.279085 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.278869 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:41.778855794 +0000 UTC m=+47.680734322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : secret "insights-runtime-extractor-tls" not found Jul 07 19:21:41.279085 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278938 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-crio-socket\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.279085 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.278936 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-data-volume\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.290611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.290580 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:41.292351 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.292329 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvvxk\" (UniqueName: \"kubernetes.io/projected/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-api-access-dvvxk\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.495218 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.495182 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:41.590647 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.590534 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:41.690616 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.690583 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:41.783488 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.783461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.783876 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.783536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:41.783876 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.783587 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 19:21:41.783876 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.783647 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:42.78363252 +0000 UTC m=+48.685511032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : configmap "kube-rbac-proxy" not found Jul 07 19:21:41.783876 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.783680 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 19:21:41.783876 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.783736 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:42.783722611 +0000 UTC m=+48.685601147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : secret "insights-runtime-extractor-tls" not found Jul 07 19:21:41.799460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.799438 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:41.799580 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.799503 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:21:41.894972 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.894890 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:41.924226 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.924199 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs"] Jul 07 19:21:41.945010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.944984 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs"] Jul 07 19:21:41.945169 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.945125 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" Jul 07 19:21:41.948101 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.948081 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 19:21:41.948998 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.948979 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-s6wmq\"" Jul 07 19:21:41.949068 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.948979 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 19:21:41.964103 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.964084 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:21:41.964349 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.964338 2561 scope.go:117] "RemoveContainer" containerID="37fcf83c0fa4d2eb3e0df0d6c79916ecf3a511c5eef1566ee6791a8d18cdee60" Jul 07 19:21:41.964519 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:41.964502 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tbdsk_openshift-console-operator(ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5)\"" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" podUID="ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5" Jul 07 19:21:41.985075 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:41.985048 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97ntc\" (UniqueName: \"kubernetes.io/projected/f762579c-08e3-472c-9a89-ca4fe1402c42-kube-api-access-97ntc\") pod \"migrator-d57d58cc9-q68bs\" (UID: \"f762579c-08e3-472c-9a89-ca4fe1402c42\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" Jul 07 19:21:42.003287 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.003261 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:42.076027 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.075997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:42.086200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.086177 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97ntc\" (UniqueName: \"kubernetes.io/projected/f762579c-08e3-472c-9a89-ca4fe1402c42-kube-api-access-97ntc\") pod \"migrator-d57d58cc9-q68bs\" (UID: \"f762579c-08e3-472c-9a89-ca4fe1402c42\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" Jul 07 19:21:42.098655 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.098634 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97ntc\" (UniqueName: \"kubernetes.io/projected/f762579c-08e3-472c-9a89-ca4fe1402c42-kube-api-access-97ntc\") pod \"migrator-d57d58cc9-q68bs\" (UID: \"f762579c-08e3-472c-9a89-ca4fe1402c42\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" Jul 07 19:21:42.117165 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.117144 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fm4q2_ad798b99-e8b2-439e-ba56-c9805dfacd9d/dns-node-resolver/0.log" Jul 07 19:21:42.254051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.254019 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" Jul 07 19:21:42.409818 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.409792 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs"] Jul 07 19:21:42.412767 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:42.412740 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf762579c_08e3_472c_9a89_ca4fe1402c42.slice/crio-8adb8443730c1f36ecae8ab6310dd977cd2550d62a75b8f6cb7a3f1686b31009 WatchSource:0}: Error finding container 8adb8443730c1f36ecae8ab6310dd977cd2550d62a75b8f6cb7a3f1686b31009: Status 404 returned error can't find the container with id 8adb8443730c1f36ecae8ab6310dd977cd2550d62a75b8f6cb7a3f1686b31009 Jul 07 19:21:42.793051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.793011 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:42.794205 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.793151 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:42.794205 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:42.793331 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 19:21:42.794205 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:42.793433 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:44.793411555 +0000 UTC m=+50.695290092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : secret "insights-runtime-extractor-tls" not found Jul 07 19:21:42.795265 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:42.795239 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 19:21:42.795371 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:42.795311 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:44.795293518 +0000 UTC m=+50.697172047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : configmap "kube-rbac-proxy" not found Jul 07 19:21:42.968948 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.968909 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" event={"ID":"f762579c-08e3-472c-9a89-ca4fe1402c42","Type":"ContainerStarted","Data":"8adb8443730c1f36ecae8ab6310dd977cd2550d62a75b8f6cb7a3f1686b31009"} Jul 07 19:21:42.969256 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:42.969228 2561 scope.go:117] "RemoveContainer" containerID="37fcf83c0fa4d2eb3e0df0d6c79916ecf3a511c5eef1566ee6791a8d18cdee60" Jul 07 19:21:42.969451 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:42.969431 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tbdsk_openshift-console-operator(ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5)\"" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" podUID="ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5" Jul 07 19:21:43.117263 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.117186 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4w272_0964a1be-fa18-4329-a4c6-e93935334f52/node-ca/0.log" Jul 07 19:21:43.441105 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.441076 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-qtqtj"] Jul 07 19:21:43.457105 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.457077 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-qtqtj"] Jul 07 19:21:43.457225 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.457175 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.459954 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.459934 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 19:21:43.460076 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.459955 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 19:21:43.460948 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.460929 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 19:21:43.461052 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.460958 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-flfc2\"" Jul 07 19:21:43.461052 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.460960 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 19:21:43.488602 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.488569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:43.498663 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.498645 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a11160ab-28a1-4f05-b665-9037dd4fe354-signing-cabundle\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.498781 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.498766 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fjdp\" (UniqueName: \"kubernetes.io/projected/a11160ab-28a1-4f05-b665-9037dd4fe354-kube-api-access-2fjdp\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.498825 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.498798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a11160ab-28a1-4f05-b665-9037dd4fe354-signing-key\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.576151 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.576125 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:43.599459 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.599432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a11160ab-28a1-4f05-b665-9037dd4fe354-signing-cabundle\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.599555 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.599539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fjdp\" (UniqueName: \"kubernetes.io/projected/a11160ab-28a1-4f05-b665-9037dd4fe354-kube-api-access-2fjdp\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.599605 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.599572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a11160ab-28a1-4f05-b665-9037dd4fe354-signing-key\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.600093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.600072 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a11160ab-28a1-4f05-b665-9037dd4fe354-signing-cabundle\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.601872 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.601852 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a11160ab-28a1-4f05-b665-9037dd4fe354-signing-key\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.609343 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.609317 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fjdp\" (UniqueName: \"kubernetes.io/projected/a11160ab-28a1-4f05-b665-9037dd4fe354-kube-api-access-2fjdp\") pod \"service-ca-758f599c5-qtqtj\" (UID: \"a11160ab-28a1-4f05-b665-9037dd4fe354\") " pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.765929 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.765855 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-qtqtj" Jul 07 19:21:43.916134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:43.916105 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdbsg_3ecf5971-da77-445e-b564-b403dcdae2ee/serve-healthcheck-canary/0.log" Jul 07 19:21:44.012136 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.012109 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-qtqtj"] Jul 07 19:21:44.016477 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:44.016416 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11160ab_28a1_4f05_b665_9037dd4fe354.slice/crio-0964906fcda500bbe4f485649dc030d248b3789cff22c548d15bd6387324631a WatchSource:0}: Error finding container 0964906fcda500bbe4f485649dc030d248b3789cff22c548d15bd6387324631a: Status 404 returned error can't find the container with id 0964906fcda500bbe4f485649dc030d248b3789cff22c548d15bd6387324631a Jul 07 19:21:44.501441 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.501409 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:44.810014 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.809976 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:44.810208 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.810111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:44.810208 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:44.810128 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 19:21:44.810208 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:44.810186 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:48.8101719 +0000 UTC m=+54.712050412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : secret "insights-runtime-extractor-tls" not found Jul 07 19:21:44.810208 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:44.810191 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 19:21:44.810366 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:44.810240 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:48.810224457 +0000 UTC m=+54.712102988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : configmap "kube-rbac-proxy" not found Jul 07 19:21:44.981035 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.980990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" event={"ID":"f762579c-08e3-472c-9a89-ca4fe1402c42","Type":"ContainerStarted","Data":"753717c1a71c460e6d0da2c45d5d13dbb03f6e2e75ee9641e6409c8c5fab320c"} Jul 07 19:21:44.981494 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.981072 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" event={"ID":"f762579c-08e3-472c-9a89-ca4fe1402c42","Type":"ContainerStarted","Data":"a60ef3ec9c04e7f730f5982757fccf37f8755eaab012857f0bfcb8ec90ad067f"} Jul 07 19:21:44.982543 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.982509 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-qtqtj" event={"ID":"a11160ab-28a1-4f05-b665-9037dd4fe354","Type":"ContainerStarted","Data":"6ae96a6fb0ad3357d8fe462be163ef3af97ed22a876850525f27883757c86681"} Jul 07 19:21:44.982671 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:44.982556 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-qtqtj" event={"ID":"a11160ab-28a1-4f05-b665-9037dd4fe354","Type":"ContainerStarted","Data":"0964906fcda500bbe4f485649dc030d248b3789cff22c548d15bd6387324631a"} Jul 07 19:21:45.033302 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.033242 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-qtqtj" podStartSLOduration=2.033219673 podStartE2EDuration="2.033219673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:45.031531034 +0000 UTC m=+50.933409573" watchObservedRunningTime="2026-07-07 19:21:45.033219673 +0000 UTC m=+50.935098210" Jul 07 19:21:45.033540 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.033501 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-q68bs" podStartSLOduration=2.199018665 podStartE2EDuration="4.033486292s" totalImagesPullingTime="1.834467627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:45.006801318 +0000 UTC m=+50.908679854" watchObservedRunningTime="2026-07-07 19:21:45.033486292 +0000 UTC m=+50.935364822" Jul 07 19:21:45.073495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.073418 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:45.281870 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.281832 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:45.427699 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.427616 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:45.526506 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.526472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:45.806173 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.806137 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:45.941328 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:45.941296 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:46.360853 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:46.360822 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:46.984830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:46.984797 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:47.332197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.332111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:47.332197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.332144 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:21:47.332197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.332166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:21:47.332197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.332190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.332251 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332283 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332315 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:03.332294436 +0000 UTC m=+69.234172952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : configmap references non-existent config key: service-ca.crt Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332343 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs podName:00bb5516-a12a-4606-b281-d2cb60b6a0f8 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:03.332333652 +0000 UTC m=+69.234212179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs") pod "router-default-6f4d8db47c-k2k2k" (UID: "00bb5516-a12a-4606-b281-d2cb60b6a0f8") : secret "router-metrics-certs-default" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.332283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332356 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332402 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332283 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332416 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78db467dbd-64sfj: secret "image-registry-tls" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332422 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls podName:4983fdb8-9403-40d2-bc8e-18ecd0ee6b74 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:03.332406982 +0000 UTC m=+69.234285504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-26j4c" (UID: "4983fdb8-9403-40d2-bc8e-18ecd0ee6b74") : secret "cluster-monitoring-operator-tls" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332444 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls podName:9207323e-9118-4aba-bf0d-47b77fd13358 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:03.332435371 +0000 UTC m=+69.234313887 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls") pod "dns-default-v8bmn" (UID: "9207323e-9118-4aba-bf0d-47b77fd13358") : secret "dns-default-metrics-tls" not found Jul 07 19:21:47.332541 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:47.332459 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls podName:c36715ae-d4d5-4567-b01d-7997b87573ac nodeName:}" failed. No retries permitted until 2026-07-07 19:22:03.332451214 +0000 UTC m=+69.234329733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls") pod "image-registry-78db467dbd-64sfj" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac") : secret "image-registry-tls" not found Jul 07 19:21:47.335270 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.335250 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f22b39b2-3315-48e7-b2a4-70ead9b74898-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-gzfc6\" (UID: \"f22b39b2-3315-48e7-b2a4-70ead9b74898\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:47.381138 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.381108 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" Jul 07 19:21:47.424865 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.424825 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:47.519541 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.519493 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6"] Jul 07 19:21:47.764249 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.764211 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:47.875955 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.875916 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:47.992075 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.992041 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:47.994012 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:47.993990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" event={"ID":"f22b39b2-3315-48e7-b2a4-70ead9b74898","Type":"ContainerStarted","Data":"70ed6a2193bfa8fe449a812226f95736da14d79b69df1b38a620a88296f1c069"} Jul 07 19:21:48.242006 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:48.241974 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:48.595203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:48.595103 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:48.749142 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:48.749103 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:48.847989 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:48.847904 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:48.848145 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:48.848024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:48.848145 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:48.848045 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 19:21:48.848145 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:48.848107 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:56.848088363 +0000 UTC m=+62.749966879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : secret "insights-runtime-extractor-tls" not found Jul 07 19:21:48.848275 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:48.848157 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 19:21:48.848275 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:21:48.848222 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm podName:6ea968bb-4c50-4b66-ba6c-cfef43a66b23 nodeName:}" failed. No retries permitted until 2026-07-07 19:21:56.848203597 +0000 UTC m=+62.750082121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t727j" (UID: "6ea968bb-4c50-4b66-ba6c-cfef43a66b23") : configmap "kube-rbac-proxy" not found Jul 07 19:21:48.966298 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:48.966242 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:49.149703 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:49.149622 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:49.249734 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:49.249661 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:49.668078 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:49.667997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:49.769432 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:49.769367 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:50.003918 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:50.003887 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" event={"ID":"f22b39b2-3315-48e7-b2a4-70ead9b74898","Type":"ContainerStarted","Data":"3ad75ca15d5f7f286e8bc6ff43f3e638ae8d17f20a2dcaa037b04c956424beaa"} Jul 07 19:21:50.004073 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:50.003936 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" event={"ID":"f22b39b2-3315-48e7-b2a4-70ead9b74898","Type":"ContainerStarted","Data":"6ac008114b4c1818f8d63f236c4c6394fc0042765223d7d5e3bc5fc60c1132b2"} Jul 07 19:21:50.022611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:50.022555 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-gzfc6" podStartSLOduration=25.19056054 podStartE2EDuration="27.022540981s" totalImagesPullingTime="1.831980441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:50.022135568 +0000 UTC m=+55.924014103" watchObservedRunningTime="2026-07-07 19:21:50.022540981 +0000 UTC m=+55.924419494" Jul 07 19:21:50.208818 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:50.208783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:50.816849 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:50.816814 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:50.979172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:50.979138 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:51.072974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:51.072901 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:51.172945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:51.172913 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:51.553439 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:51.553406 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:51.628059 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:51.628031 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:51.947164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:51.947130 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:52.786683 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:52.786653 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:52.946644 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:52.946599 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:53.342694 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:53.342659 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:53.538399 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:53.538343 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:54.151727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:54.151698 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:54.253590 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:54.253559 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:54.746405 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:54.746360 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:54.947574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:54.947534 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:55.144507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:55.144421 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:55.539183 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:55.539150 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:55.783965 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:55.783913 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:55.961645 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:55.961603 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:56.918176 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:56.918140 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:56.918597 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:56.918242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:56.918764 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:56.918741 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:56.920430 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:56.920407 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ea968bb-4c50-4b66-ba6c-cfef43a66b23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t727j\" (UID: \"6ea968bb-4c50-4b66-ba6c-cfef43a66b23\") " pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:57.001113 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:57.001089 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ktbrl\"" Jul 07 19:21:57.009347 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:57.009327 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t727j" Jul 07 19:21:57.133686 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:57.133605 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t727j"] Jul 07 19:21:57.136096 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:21:57.136076 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea968bb_4c50_4b66_ba6c_cfef43a66b23.slice/crio-d75d1b2ef7eb4ca5c5966447103dcac768340e219402033ce85997aa75672945 WatchSource:0}: Error finding container d75d1b2ef7eb4ca5c5966447103dcac768340e219402033ce85997aa75672945: Status 404 returned error can't find the container with id d75d1b2ef7eb4ca5c5966447103dcac768340e219402033ce85997aa75672945 Jul 07 19:21:57.618403 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:57.618113 2561 scope.go:117] "RemoveContainer" containerID="37fcf83c0fa4d2eb3e0df0d6c79916ecf3a511c5eef1566ee6791a8d18cdee60" Jul 07 19:21:57.845094 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:57.845066 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrh2c" Jul 07 19:21:58.036408 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.036129 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:21:58.036408 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.036274 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" event={"ID":"ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5","Type":"ContainerStarted","Data":"6eb48f3420b700e780edba6b8d42967391e80fd42c2675b69a5fe2c5c029bcde"} Jul 07 19:21:58.038224 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.038196 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t727j" event={"ID":"6ea968bb-4c50-4b66-ba6c-cfef43a66b23","Type":"ContainerStarted","Data":"d75d1b2ef7eb4ca5c5966447103dcac768340e219402033ce85997aa75672945"} Jul 07 19:21:58.066353 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.066299 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" podStartSLOduration=25.616602668 podStartE2EDuration="33.066278087s" totalImagesPullingTime="7.449675419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:21:58.065004335 +0000 UTC m=+63.966882899" watchObservedRunningTime="2026-07-07 19:21:58.066278087 +0000 UTC m=+63.968156623" Jul 07 19:21:58.506724 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.506673 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:58.606973 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.606940 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:58.670205 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.670166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:58.796498 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:58.796410 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:59.186226 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.186188 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:59.250864 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.250826 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:59.341453 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.341416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:59.344286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.344255 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 19:21:59.354611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.354575 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92f94e51-0ce7-4298-95ea-b6df5e03d944-metrics-certs\") pod \"network-metrics-daemon-cvqcz\" (UID: \"92f94e51-0ce7-4298-95ea-b6df5e03d944\") " pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:59.389079 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.389045 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:59.442508 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.442428 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:59.442508 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.442504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:59.445294 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.445265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvb2q\" (UniqueName: \"kubernetes.io/projected/fd4b129d-5629-40e3-b586-3e89d67d703b-kube-api-access-pvb2q\") pod \"network-check-target-88vw9\" (UID: \"fd4b129d-5629-40e3-b586-3e89d67d703b\") " pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:59.445434 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.445271 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 19:21:59.455515 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.455495 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b23202de-78de-407f-b354-2c83ecd07eed-original-pull-secret\") pod \"global-pull-secret-syncer-j474f\" (UID: \"b23202de-78de-407f-b354-2c83ecd07eed\") " pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:59.485855 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.485829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:59.528698 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.528659 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rq87d\"" Jul 07 19:21:59.533964 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.533940 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhh8r\"" Jul 07 19:21:59.536021 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.535993 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvqcz" Jul 07 19:21:59.537164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.537144 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j474f" Jul 07 19:21:59.542301 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.542279 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:21:59.841079 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.841004 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:21:59.973853 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:21:59.973810 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:00.077404 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:00.077349 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:00.147483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:00.147405 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:00.235484 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:00.235449 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:00.981883 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:00.981849 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f94e51_0ce7_4298_95ea_b6df5e03d944.slice/crio-bca50ac979ebc40df7fd98dc19fc90c897456034c5a5954c6b4625989073ef3f WatchSource:0}: Error finding container bca50ac979ebc40df7fd98dc19fc90c897456034c5a5954c6b4625989073ef3f: Status 404 returned error can't find the container with id bca50ac979ebc40df7fd98dc19fc90c897456034c5a5954c6b4625989073ef3f Jul 07 19:22:00.981883 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:00.981872 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cvqcz"] Jul 07 19:22:00.993654 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:00.993634 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-88vw9"] Jul 07 19:22:00.996252 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:00.996222 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4b129d_5629_40e3_b586_3e89d67d703b.slice/crio-4a58f3633671b4bcc9ba6a8b910379a4608fbbe4bb8c5260573c382bc444ad5c WatchSource:0}: Error finding container 4a58f3633671b4bcc9ba6a8b910379a4608fbbe4bb8c5260573c382bc444ad5c: Status 404 returned error can't find the container with id 4a58f3633671b4bcc9ba6a8b910379a4608fbbe4bb8c5260573c382bc444ad5c Jul 07 19:22:01.014323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:01.014269 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j474f"] Jul 07 19:22:01.017831 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:01.017805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:01.023444 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:01.023416 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23202de_78de_407f_b354_2c83ecd07eed.slice/crio-51c8d85dddc887c6978fc74b3f4978f6128a27e9ec5d81321082aa852d138c2d WatchSource:0}: Error finding container 51c8d85dddc887c6978fc74b3f4978f6128a27e9ec5d81321082aa852d138c2d: Status 404 returned error can't find the container with id 51c8d85dddc887c6978fc74b3f4978f6128a27e9ec5d81321082aa852d138c2d Jul 07 19:22:01.049165 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:01.049141 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j474f" event={"ID":"b23202de-78de-407f-b354-2c83ecd07eed","Type":"ContainerStarted","Data":"51c8d85dddc887c6978fc74b3f4978f6128a27e9ec5d81321082aa852d138c2d"} Jul 07 19:22:01.050721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:01.050699 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t727j" event={"ID":"6ea968bb-4c50-4b66-ba6c-cfef43a66b23","Type":"ContainerStarted","Data":"e4605b3c00083b70aaa87ae5b907bdd2b33a1971dff7f46ede217af584b8233a"} Jul 07 19:22:01.051867 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:01.051843 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cvqcz" event={"ID":"92f94e51-0ce7-4298-95ea-b6df5e03d944","Type":"ContainerStarted","Data":"bca50ac979ebc40df7fd98dc19fc90c897456034c5a5954c6b4625989073ef3f"} Jul 07 19:22:01.052884 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:01.052864 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-88vw9" event={"ID":"fd4b129d-5629-40e3-b586-3e89d67d703b","Type":"ContainerStarted","Data":"4a58f3633671b4bcc9ba6a8b910379a4608fbbe4bb8c5260573c382bc444ad5c"} Jul 07 19:22:02.059408 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:02.059361 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t727j" event={"ID":"6ea968bb-4c50-4b66-ba6c-cfef43a66b23","Type":"ContainerStarted","Data":"b5e43e632e9362b106efe159bb0454dbbca03c2ecea23b37a0415676a8271871"} Jul 07 19:22:02.064168 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:02.064133 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-88vw9" event={"ID":"fd4b129d-5629-40e3-b586-3e89d67d703b","Type":"ContainerStarted","Data":"2888c1d71a22835140bfee270a23ea59099ff93b59a8e2f5a1d30aad9fcf99d1"} Jul 07 19:22:02.064520 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:02.064499 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:22:02.084242 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:02.084190 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-88vw9" podStartSLOduration=68.084174309 podStartE2EDuration="1m8.084174309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:02.083554513 +0000 UTC m=+67.985433050" watchObservedRunningTime="2026-07-07 19:22:02.084174309 +0000 UTC m=+67.986052840" Jul 07 19:22:03.071749 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.071708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cvqcz" event={"ID":"92f94e51-0ce7-4298-95ea-b6df5e03d944","Type":"ContainerStarted","Data":"e0f2da75a28e0f823e265e6679148a25a3cf0c29801904fe2a251490d56d57a8"} Jul 07 19:22:03.072229 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.071778 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cvqcz" event={"ID":"92f94e51-0ce7-4298-95ea-b6df5e03d944","Type":"ContainerStarted","Data":"d1bec34d81f596f78da9dc0fcc326e4e31d277b7569bf1aa131dfe87547afd03"} Jul 07 19:22:03.096219 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.095869 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cvqcz" podStartSLOduration=67.576861326 podStartE2EDuration="1m9.095849725s" totalImagesPullingTime="1.518988399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:03.095274223 +0000 UTC m=+68.997152761" watchObservedRunningTime="2026-07-07 19:22:03.095849725 +0000 UTC m=+68.997728263" Jul 07 19:22:03.382089 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.382007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:03.382089 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.382060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:22:03.382310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.382094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:03.382498 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.382460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:03.382611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.382512 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:22:03.382884 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.382862 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00bb5516-a12a-4606-b281-d2cb60b6a0f8-service-ca-bundle\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:03.385286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.384914 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9207323e-9118-4aba-bf0d-47b77fd13358-metrics-tls\") pod \"dns-default-v8bmn\" (UID: \"9207323e-9118-4aba-bf0d-47b77fd13358\") " pod="openshift-dns/dns-default-v8bmn" Jul 07 19:22:03.385371 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.385312 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00bb5516-a12a-4606-b281-d2cb60b6a0f8-metrics-certs\") pod \"router-default-6f4d8db47c-k2k2k\" (UID: \"00bb5516-a12a-4606-b281-d2cb60b6a0f8\") " pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:03.385459 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.385413 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"image-registry-78db467dbd-64sfj\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:03.385919 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.385898 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4983fdb8-9403-40d2-bc8e-18ecd0ee6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-26j4c\" (UID: \"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:22:03.539028 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.538954 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jrgxk\"" Jul 07 19:22:03.545993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.545791 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6t2z8\"" Jul 07 19:22:03.546824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.546800 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:03.553976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.553955 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" Jul 07 19:22:03.571232 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.571210 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cphpf\"" Jul 07 19:22:03.578832 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.578802 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:03.610114 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.610094 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lxjvf\"" Jul 07 19:22:03.617548 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:03.617519 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8bmn" Jul 07 19:22:05.109516 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.108566 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx"] Jul 07 19:22:05.111500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.111479 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7"] Jul 07 19:22:05.111647 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.111617 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.114286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.114261 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5"] Jul 07 19:22:05.114431 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.114393 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.117519 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.117500 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn"] Jul 07 19:22:05.117668 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.117642 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.117945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.117907 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 19:22:05.117945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.117930 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 19:22:05.118099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.118060 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2hd8v\"" Jul 07 19:22:05.118099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.118066 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qwlpt\"" Jul 07 19:22:05.118099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.118092 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 19:22:05.118892 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.118465 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 19:22:05.118892 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.118485 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 19:22:05.120021 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.119955 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 19:22:05.121277 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.121258 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 19:22:05.123450 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.123431 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.125015 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.124989 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx"] Jul 07 19:22:05.128211 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.128146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 19:22:05.128443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.128425 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 19:22:05.128516 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.128443 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 19:22:05.128516 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.128485 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 19:22:05.134092 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.134067 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7"] Jul 07 19:22:05.140081 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.139699 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5"] Jul 07 19:22:05.141529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.141436 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn"] Jul 07 19:22:05.199148 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199113 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-hub\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.199325 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199163 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j9bmx\" (UID: \"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.199325 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199207 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j9bmx\" (UID: \"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.199325 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.199325 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25c32f0a-e089-4b7a-85f3-67abeeb88400-tmp\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/25c32f0a-e089-4b7a-85f3-67abeeb88400-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199401 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w7d4\" (UniqueName: \"kubernetes.io/projected/25c32f0a-e089-4b7a-85f3-67abeeb88400-kube-api-access-4w7d4\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199428 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd63a26-061c-4849-936b-c2206893f23a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199454 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-ca\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkgph\" (UniqueName: \"kubernetes.io/projected/bbd63a26-061c-4849-936b-c2206893f23a-kube-api-access-zkgph\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199526 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0961df2b-5507-46fc-b923-434a57ac323a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67985b6fc6-whnb7\" (UID: \"0961df2b-5507-46fc-b923-434a57ac323a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.199578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199550 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.199906 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.199696 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx6bm\" (UniqueName: \"kubernetes.io/projected/0961df2b-5507-46fc-b923-434a57ac323a-kube-api-access-fx6bm\") pod \"managed-serviceaccount-addon-agent-67985b6fc6-whnb7\" (UID: \"0961df2b-5507-46fc-b923-434a57ac323a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.205017 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.204988 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-4nbcv"] Jul 07 19:22:05.207847 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.207823 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:05.213449 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.213424 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-p5rrt\"" Jul 07 19:22:05.213747 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.213729 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 19:22:05.217929 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.217889 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 19:22:05.222392 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.222350 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-4nbcv"] Jul 07 19:22:05.230373 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.230284 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:05.300526 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx6bm\" (UniqueName: \"kubernetes.io/projected/0961df2b-5507-46fc-b923-434a57ac323a-kube-api-access-fx6bm\") pod \"managed-serviceaccount-addon-agent-67985b6fc6-whnb7\" (UID: \"0961df2b-5507-46fc-b923-434a57ac323a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.300687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300536 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76xq9\" (UniqueName: \"kubernetes.io/projected/2f618da1-73e6-47a9-9411-c46148b7b563-kube-api-access-76xq9\") pod \"downloads-685bfb5f64-4nbcv\" (UID: \"2f618da1-73e6-47a9-9411-c46148b7b563\") " pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:05.300687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-hub\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.300687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300604 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j9bmx\" (UID: \"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.300687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300674 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j9bmx\" (UID: \"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.300915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.300915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25c32f0a-e089-4b7a-85f3-67abeeb88400-tmp\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.300915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300776 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/25c32f0a-e089-4b7a-85f3-67abeeb88400-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.300915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w7d4\" (UniqueName: \"kubernetes.io/projected/25c32f0a-e089-4b7a-85f3-67abeeb88400-kube-api-access-4w7d4\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.300915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd63a26-061c-4849-936b-c2206893f23a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.300915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.300866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-ca\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.301226 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.301197 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25c32f0a-e089-4b7a-85f3-67abeeb88400-tmp\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.301638 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.301344 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j9bmx\" (UID: \"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.301638 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.301434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkgph\" (UniqueName: \"kubernetes.io/projected/bbd63a26-061c-4849-936b-c2206893f23a-kube-api-access-zkgph\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.301638 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.301482 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0961df2b-5507-46fc-b923-434a57ac323a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67985b6fc6-whnb7\" (UID: \"0961df2b-5507-46fc-b923-434a57ac323a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.301638 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.301510 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.302266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.302214 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd63a26-061c-4849-936b-c2206893f23a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.304318 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.304287 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j9bmx\" (UID: \"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.304566 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.304542 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/25c32f0a-e089-4b7a-85f3-67abeeb88400-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.304823 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.304801 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-hub\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.304895 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.304879 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0961df2b-5507-46fc-b923-434a57ac323a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67985b6fc6-whnb7\" (UID: \"0961df2b-5507-46fc-b923-434a57ac323a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.304951 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.304881 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-ca\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.305137 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.305115 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.305754 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.305737 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bbd63a26-061c-4849-936b-c2206893f23a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.310580 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.310557 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w7d4\" (UniqueName: \"kubernetes.io/projected/25c32f0a-e089-4b7a-85f3-67abeeb88400-kube-api-access-4w7d4\") pod \"klusterlet-addon-workmgr-7d9f88df9f-r7px5\" (UID: \"25c32f0a-e089-4b7a-85f3-67abeeb88400\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.311525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.311493 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkgph\" (UniqueName: \"kubernetes.io/projected/bbd63a26-061c-4849-936b-c2206893f23a-kube-api-access-zkgph\") pod \"cluster-proxy-proxy-agent-855c66bc99-k4pdn\" (UID: \"bbd63a26-061c-4849-936b-c2206893f23a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.311725 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.311694 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx6bm\" (UniqueName: \"kubernetes.io/projected/0961df2b-5507-46fc-b923-434a57ac323a-kube-api-access-fx6bm\") pod \"managed-serviceaccount-addon-agent-67985b6fc6-whnb7\" (UID: \"0961df2b-5507-46fc-b923-434a57ac323a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.317276 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.317245 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:05.402828 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.402743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xq9\" (UniqueName: \"kubernetes.io/projected/2f618da1-73e6-47a9-9411-c46148b7b563-kube-api-access-76xq9\") pod \"downloads-685bfb5f64-4nbcv\" (UID: \"2f618da1-73e6-47a9-9411-c46148b7b563\") " pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:05.412739 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.412711 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xq9\" (UniqueName: \"kubernetes.io/projected/2f618da1-73e6-47a9-9411-c46148b7b563-kube-api-access-76xq9\") pod \"downloads-685bfb5f64-4nbcv\" (UID: \"2f618da1-73e6-47a9-9411-c46148b7b563\") " pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:05.423220 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.423192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:05.425622 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.425598 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" Jul 07 19:22:05.439983 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.439965 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" Jul 07 19:22:05.450042 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.450023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:05.456312 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.456291 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" Jul 07 19:22:05.515729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.515684 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:05.529277 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:05.529203 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:06.032167 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.032097 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6f4d8db47c-k2k2k"] Jul 07 19:22:06.034267 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.034236 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bb5516_a12a_4606_b281_d2cb60b6a0f8.slice/crio-664dc50ae27d1a58dc0be40f99c9ec0ae408280c9f7a4d6e5f0f19f1738f4417 WatchSource:0}: Error finding container 664dc50ae27d1a58dc0be40f99c9ec0ae408280c9f7a4d6e5f0f19f1738f4417: Status 404 returned error can't find the container with id 664dc50ae27d1a58dc0be40f99c9ec0ae408280c9f7a4d6e5f0f19f1738f4417 Jul 07 19:22:06.092491 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.091934 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8bmn"] Jul 07 19:22:06.094400 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.094311 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" event={"ID":"00bb5516-a12a-4606-b281-d2cb60b6a0f8","Type":"ContainerStarted","Data":"664dc50ae27d1a58dc0be40f99c9ec0ae408280c9f7a4d6e5f0f19f1738f4417"} Jul 07 19:22:06.096214 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.096188 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j474f" event={"ID":"b23202de-78de-407f-b354-2c83ecd07eed","Type":"ContainerStarted","Data":"960075fc30bc5bc73915ba0fe90006acfb41982e5ee6d5114eca9f7ae0fbfb1c"} Jul 07 19:22:06.096486 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.096340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:06.099117 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.098601 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t727j" event={"ID":"6ea968bb-4c50-4b66-ba6c-cfef43a66b23","Type":"ContainerStarted","Data":"a25bcf835326a9ccba390682d9ac0fe0e42aa215b8a06b79a680a0d7b529f3fe"} Jul 07 19:22:06.122200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.121759 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j474f" podStartSLOduration=66.344316815 podStartE2EDuration="1m11.121737333s" totalImagesPullingTime="4.777420518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:06.116791314 +0000 UTC m=+72.018669849" watchObservedRunningTime="2026-07-07 19:22:06.121737333 +0000 UTC m=+72.023615860" Jul 07 19:22:06.147058 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.146554 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t727j" podStartSLOduration=17.672903926 podStartE2EDuration="26.146535387s" totalImagesPullingTime="8.473631461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:21:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:06.146218248 +0000 UTC m=+72.048096797" watchObservedRunningTime="2026-07-07 19:22:06.146535387 +0000 UTC m=+72.048413921" Jul 07 19:22:06.192089 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.192063 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn"] Jul 07 19:22:06.195146 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.195114 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbd63a26_061c_4849_936b_c2206893f23a.slice/crio-22457176ab5ca435952e050ef4e4d2de4668a67f6f250a85eb6b101dffaf8f09 WatchSource:0}: Error finding container 22457176ab5ca435952e050ef4e4d2de4668a67f6f250a85eb6b101dffaf8f09: Status 404 returned error can't find the container with id 22457176ab5ca435952e050ef4e4d2de4668a67f6f250a85eb6b101dffaf8f09 Jul 07 19:22:06.195627 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.195594 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:06.278981 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.278958 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c"] Jul 07 19:22:06.281460 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.281435 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4983fdb8_9403_40d2_bc8e_18ecd0ee6b74.slice/crio-a91fa17ecf097972e594529802c39de60f94928c3652dcf8af9abc7deb70a122 WatchSource:0}: Error finding container a91fa17ecf097972e594529802c39de60f94928c3652dcf8af9abc7deb70a122: Status 404 returned error can't find the container with id a91fa17ecf097972e594529802c39de60f94928c3652dcf8af9abc7deb70a122 Jul 07 19:22:06.281628 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.281609 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5"] Jul 07 19:22:06.284266 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.284248 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c32f0a_e089_4b7a_85f3_67abeeb88400.slice/crio-3eb23abc8ced0516fb28189f7bffd8a236dced5e8add1e13b6e87c8241f46a33 WatchSource:0}: Error finding container 3eb23abc8ced0516fb28189f7bffd8a236dced5e8add1e13b6e87c8241f46a33: Status 404 returned error can't find the container with id 3eb23abc8ced0516fb28189f7bffd8a236dced5e8add1e13b6e87c8241f46a33 Jul 07 19:22:06.322954 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.322930 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7"] Jul 07 19:22:06.326082 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.326048 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx"] Jul 07 19:22:06.326229 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.326154 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0961df2b_5507_46fc_b923_434a57ac323a.slice/crio-8edee7c5beb94fedd68849bf5c39b552cdc73213abd9a634d3b162ccd674c598 WatchSource:0}: Error finding container 8edee7c5beb94fedd68849bf5c39b552cdc73213abd9a634d3b162ccd674c598: Status 404 returned error can't find the container with id 8edee7c5beb94fedd68849bf5c39b552cdc73213abd9a634d3b162ccd674c598 Jul 07 19:22:06.331013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.330989 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-4nbcv"] Jul 07 19:22:06.332432 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:06.332357 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78db467dbd-64sfj"] Jul 07 19:22:06.337719 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.337663 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36715ae_d4d5_4567_b01d_7997b87573ac.slice/crio-612efc1d404733b8dbe55b21d69fd88f25d39fbae573eaf52441aa85dbddbe06 WatchSource:0}: Error finding container 612efc1d404733b8dbe55b21d69fd88f25d39fbae573eaf52441aa85dbddbe06: Status 404 returned error can't find the container with id 612efc1d404733b8dbe55b21d69fd88f25d39fbae573eaf52441aa85dbddbe06 Jul 07 19:22:06.338791 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:06.338678 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f618da1_73e6_47a9_9411_c46148b7b563.slice/crio-edf0dbd55f75f66880b939f3b58e93fdbd3917d1462d0beada824011f87b46d3 WatchSource:0}: Error finding container edf0dbd55f75f66880b939f3b58e93fdbd3917d1462d0beada824011f87b46d3: Status 404 returned error can't find the container with id edf0dbd55f75f66880b939f3b58e93fdbd3917d1462d0beada824011f87b46d3 Jul 07 19:22:07.105584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.105474 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" event={"ID":"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb","Type":"ContainerStarted","Data":"5e2fccec764712c231dd2428e3b70115bd519c7b9e0051da324b9df0105121ca"} Jul 07 19:22:07.107244 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.107211 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-4nbcv" event={"ID":"2f618da1-73e6-47a9-9411-c46148b7b563","Type":"ContainerStarted","Data":"edf0dbd55f75f66880b939f3b58e93fdbd3917d1462d0beada824011f87b46d3"} Jul 07 19:22:07.109210 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.109185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" event={"ID":"c36715ae-d4d5-4567-b01d-7997b87573ac","Type":"ContainerStarted","Data":"d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5"} Jul 07 19:22:07.109331 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.109224 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" event={"ID":"c36715ae-d4d5-4567-b01d-7997b87573ac","Type":"ContainerStarted","Data":"612efc1d404733b8dbe55b21d69fd88f25d39fbae573eaf52441aa85dbddbe06"} Jul 07 19:22:07.109517 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.109491 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:07.111014 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.110991 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8bmn" event={"ID":"9207323e-9118-4aba-bf0d-47b77fd13358","Type":"ContainerStarted","Data":"dc073dfd064c7ca4f19528a6763eef7b78ed24ac288233c1fb1cb7bf5a2705c4"} Jul 07 19:22:07.112861 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.112827 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" event={"ID":"25c32f0a-e089-4b7a-85f3-67abeeb88400","Type":"ContainerStarted","Data":"3eb23abc8ced0516fb28189f7bffd8a236dced5e8add1e13b6e87c8241f46a33"} Jul 07 19:22:07.115333 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.115310 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" event={"ID":"00bb5516-a12a-4606-b281-d2cb60b6a0f8","Type":"ContainerStarted","Data":"9d907b4192c945d8548e32daea0dead845b59296823be0d093d492c283b736f1"} Jul 07 19:22:07.117778 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.117755 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" event={"ID":"bbd63a26-061c-4849-936b-c2206893f23a","Type":"ContainerStarted","Data":"22457176ab5ca435952e050ef4e4d2de4668a67f6f250a85eb6b101dffaf8f09"} Jul 07 19:22:07.118950 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.118926 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" event={"ID":"0961df2b-5507-46fc-b923-434a57ac323a","Type":"ContainerStarted","Data":"8edee7c5beb94fedd68849bf5c39b552cdc73213abd9a634d3b162ccd674c598"} Jul 07 19:22:07.120283 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.120215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" event={"ID":"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74","Type":"ContainerStarted","Data":"a91fa17ecf097972e594529802c39de60f94928c3652dcf8af9abc7deb70a122"} Jul 07 19:22:07.134805 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.134570 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" podStartSLOduration=73.134553019 podStartE2EDuration="1m13.134553019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:07.133107864 +0000 UTC m=+73.034986402" watchObservedRunningTime="2026-07-07 19:22:07.134553019 +0000 UTC m=+73.036431557" Jul 07 19:22:07.156529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.156264 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" podStartSLOduration=49.15624461 podStartE2EDuration="49.15624461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:07.153429432 +0000 UTC m=+73.055307993" watchObservedRunningTime="2026-07-07 19:22:07.15624461 +0000 UTC m=+73.058123146" Jul 07 19:22:07.579116 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.579088 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:07.581731 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:07.581707 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:08.042007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.037144 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:22:08.047509 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.047256 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-tbdsk" Jul 07 19:22:08.131576 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.131018 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" event={"ID":"e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb","Type":"ContainerStarted","Data":"8bf02c465bb1b39b997164cc89a01a7f67a7aa58de118796721a97ac572c5e0a"} Jul 07 19:22:08.131576 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.131533 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:08.133213 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.133191 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6f4d8db47c-k2k2k" Jul 07 19:22:08.154439 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.154010 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j9bmx" podStartSLOduration=1.914526204 podStartE2EDuration="3.153990091s" totalImagesPullingTime="1.239463887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:08.153023868 +0000 UTC m=+74.054902408" watchObservedRunningTime="2026-07-07 19:22:08.153990091 +0000 UTC m=+74.055868627" Jul 07 19:22:08.333941 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.333857 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:08.425805 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:08.425647 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:09.612597 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:09.612551 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:09.678969 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:09.678928 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:10.019102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:10.019068 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:10.085575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:10.085515 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:11.146945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.146857 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8bmn" event={"ID":"9207323e-9118-4aba-bf0d-47b77fd13358","Type":"ContainerStarted","Data":"07cd3d95cd5ea85bb14326031e493bace27a4122cd3010822c49b2b6d0945adc"} Jul 07 19:22:11.146945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.146918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8bmn" event={"ID":"9207323e-9118-4aba-bf0d-47b77fd13358","Type":"ContainerStarted","Data":"7d91f5b9a9a5384bb5a42a64de99043cb6b369bdbec12e53f31446ce2fc18402"} Jul 07 19:22:11.147487 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.147053 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v8bmn" Jul 07 19:22:11.148952 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.148923 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" event={"ID":"4983fdb8-9403-40d2-bc8e-18ecd0ee6b74","Type":"ContainerStarted","Data":"111948f7b9616192f05fbf40bcf5e617793901cbcf59836e3879334cb445f173"} Jul 07 19:22:11.212826 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.212776 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v8bmn" podStartSLOduration=35.769887365 podStartE2EDuration="40.212761193s" totalImagesPullingTime="4.442873828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:11.208325339 +0000 UTC m=+77.110203877" watchObservedRunningTime="2026-07-07 19:22:11.212761193 +0000 UTC m=+77.114639752" Jul 07 19:22:11.227443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.227408 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:11.251298 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.251253 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-26j4c" podStartSLOduration=48.806224488 podStartE2EDuration="53.251225343s" totalImagesPullingTime="4.445000855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:11.250708198 +0000 UTC m=+77.152586734" watchObservedRunningTime="2026-07-07 19:22:11.251225343 +0000 UTC m=+77.153103878" Jul 07 19:22:11.358749 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.358718 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:11.552774 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.552723 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:11.786521 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.786491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:11.886911 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:11.886735 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:12.074444 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:12.074410 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:12.240248 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:12.240214 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:12.322685 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:12.322611 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:14.260196 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:14.260161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:14.355064 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:14.355027 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:16.478740 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:16.478540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:16.640721 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:16.640687 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:18.644437 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:18.644307 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:18.832569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:18.832531 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:19.043995 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:19.043961 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:20.100668 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:20.100627 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:20.306125 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:20.306090 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:20.414183 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:20.414040 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:20.515727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:20.515689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:21.013652 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:21.013619 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:21.154341 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:21.154308 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v8bmn" Jul 07 19:22:22.743745 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:22.743633 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:22.843000 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:22.842938 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.061023 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.060907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.115632 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.113936 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p74tq"] Jul 07 19:22:23.159839 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.159804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.258639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.257205 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.259979 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.259957 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.266200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.265211 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tcdnl\"" Jul 07 19:22:23.266200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.265503 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 19:22:23.266200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.265689 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 19:22:23.266200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.265890 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 19:22:23.266200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.266058 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 19:22:23.285052 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285027 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-root\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285069 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6650d26-c035-4549-9da7-353999a1c0f2-metrics-client-ca\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-tls\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285127 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwj7k\" (UniqueName: \"kubernetes.io/projected/b6650d26-c035-4549-9da7-353999a1c0f2-kube-api-access-kwj7k\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-wtmp\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285422 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285179 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285422 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285212 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-textfile\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285422 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285247 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-sys\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.285422 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.285290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.358141 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.358027 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.386404 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.386569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-root\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.386569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386455 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6650d26-c035-4549-9da7-353999a1c0f2-metrics-client-ca\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.386569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386483 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-tls\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.386569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386512 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj7k\" (UniqueName: \"kubernetes.io/projected/b6650d26-c035-4549-9da7-353999a1c0f2-kube-api-access-kwj7k\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.386569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-wtmp\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.386569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386561 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.387041 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386567 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-root\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.387041 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-textfile\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.387041 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.386634 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-sys\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.387254 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.387219 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-sys\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.388536 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:22:23.387928 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 19:22:23.388536 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:22:23.388015 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-tls podName:b6650d26-c035-4549-9da7-353999a1c0f2 nodeName:}" failed. No retries permitted until 2026-07-07 19:22:23.887991921 +0000 UTC m=+89.789870448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-tls") pod "node-exporter-p74tq" (UID: "b6650d26-c035-4549-9da7-353999a1c0f2") : secret "node-exporter-tls" not found Jul 07 19:22:23.388536 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.388058 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6650d26-c035-4549-9da7-353999a1c0f2-metrics-client-ca\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.388536 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.388129 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-wtmp\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.388536 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.388418 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-textfile\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.388802 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.388608 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.391515 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.391461 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.401962 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.401905 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwj7k\" (UniqueName: \"kubernetes.io/projected/b6650d26-c035-4549-9da7-353999a1c0f2-kube-api-access-kwj7k\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.458736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.458699 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.552159 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.552121 2561 patch_prober.go:28] interesting pod/image-registry-78db467dbd-64sfj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 19:22:23.552348 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.552183 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 19:22:23.557483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.557454 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.669841 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.669748 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:23.910134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.892557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-tls\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:23.910134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:23.905794 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6650d26-c035-4549-9da7-353999a1c0f2-node-exporter-tls\") pod \"node-exporter-p74tq\" (UID: \"b6650d26-c035-4549-9da7-353999a1c0f2\") " pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:24.079396 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:24.079346 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:24.169600 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:24.169563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:24.175524 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:24.175502 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p74tq" Jul 07 19:22:24.285723 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:24.285689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:24.750211 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:22:24.750088 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6650d26_c035_4549_9da7_353999a1c0f2.slice/crio-3d27fa1fce351a8fcef982cfb5e47e89aaad9733ab26c1a7bb25c908a8ffe4bc WatchSource:0}: Error finding container 3d27fa1fce351a8fcef982cfb5e47e89aaad9733ab26c1a7bb25c908a8ffe4bc: Status 404 returned error can't find the container with id 3d27fa1fce351a8fcef982cfb5e47e89aaad9733ab26c1a7bb25c908a8ffe4bc Jul 07 19:22:25.211125 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:25.211085 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p74tq" event={"ID":"b6650d26-c035-4549-9da7-353999a1c0f2","Type":"ContainerStarted","Data":"3d27fa1fce351a8fcef982cfb5e47e89aaad9733ab26c1a7bb25c908a8ffe4bc"} Jul 07 19:22:25.212930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:25.212883 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-4nbcv" event={"ID":"2f618da1-73e6-47a9-9411-c46148b7b563","Type":"ContainerStarted","Data":"8d91b7437bbc817bd383a0f615454f8d915886f69a4cfd1dc8f416e4347465a4"} Jul 07 19:22:25.213249 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:25.213230 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:25.214653 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:25.214632 2561 patch_prober.go:28] interesting pod/downloads-685bfb5f64-4nbcv container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused" start-of-body= Jul 07 19:22:25.214759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:25.214689 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-4nbcv" podUID="2f618da1-73e6-47a9-9411-c46148b7b563" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused" Jul 07 19:22:25.231462 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:25.231410 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-4nbcv" podStartSLOduration=1.516992636 podStartE2EDuration="20.231372827s" totalImagesPullingTime="18.714380191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:25.230000886 +0000 UTC m=+91.131879426" watchObservedRunningTime="2026-07-07 19:22:25.231372827 +0000 UTC m=+91.133251363" Jul 07 19:22:26.071412 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:26.071350 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:26.167090 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:26.167000 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:26.222577 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:26.222546 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-4nbcv" Jul 07 19:22:26.303220 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:26.303173 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:26.403428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:26.403197 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:27.226705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:27.226656 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" event={"ID":"0961df2b-5507-46fc-b923-434a57ac323a","Type":"ContainerStarted","Data":"849cbcbe0925fd073cd97802ec83818e7acce8349a9dc4f71bf87186c28046d8"} Jul 07 19:22:27.228522 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:27.228491 2561 generic.go:356] "Generic (PLEG): container finished" podID="b6650d26-c035-4549-9da7-353999a1c0f2" containerID="45c89c693396f619ea0f1d093e6d12e81942e076091cf363cc9c25097a1503d0" exitCode=0 Jul 07 19:22:27.228657 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:27.228585 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p74tq" event={"ID":"b6650d26-c035-4549-9da7-353999a1c0f2","Type":"ContainerDied","Data":"45c89c693396f619ea0f1d093e6d12e81942e076091cf363cc9c25097a1503d0"} Jul 07 19:22:27.244387 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:27.244325 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67985b6fc6-whnb7" podStartSLOduration=1.957297711 podStartE2EDuration="22.244310042s" totalImagesPullingTime="20.287012331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:27.242810495 +0000 UTC m=+93.144689031" watchObservedRunningTime="2026-07-07 19:22:27.244310042 +0000 UTC m=+93.146188578" Jul 07 19:22:27.857706 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:27.857615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:27.973130 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:27.973000 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:28.071227 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.071177 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:28.138443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.138348 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:28.177023 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.176794 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:28.241483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.241447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p74tq" event={"ID":"b6650d26-c035-4549-9da7-353999a1c0f2","Type":"ContainerStarted","Data":"9e5bdb62b9725956587b2afcfd819bb9d80f2fb38ae59332a4a3737a0e8f74e6"} Jul 07 19:22:28.241925 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.241509 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p74tq" event={"ID":"b6650d26-c035-4549-9da7-353999a1c0f2","Type":"ContainerStarted","Data":"21079f52a70c23d8395b778a990aa0cacd92287d29e93c3a220ec0dc5d6a5aef"} Jul 07 19:22:28.269073 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.268918 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p74tq" podStartSLOduration=3.263556113 podStartE2EDuration="5.268896057s" totalImagesPullingTime="1.86818899s" totalInitContainerRuntime="137.150954ms" isStatefulPod=true podCreationTimestamp="2026-07-07 19:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:28.267012848 +0000 UTC m=+94.168891381" watchObservedRunningTime="2026-07-07 19:22:28.268896057 +0000 UTC m=+94.170774604" Jul 07 19:22:28.503862 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.503826 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:28.593196 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.593147 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:28.835986 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.835906 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:28.906208 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:28.905921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:29.383920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:29.383858 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:30.043184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:30.043131 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:30.117057 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:30.117011 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:30.338224 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:30.338082 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:32.276082 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.276046 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" event={"ID":"25c32f0a-e089-4b7a-85f3-67abeeb88400","Type":"ContainerStarted","Data":"8768e2715ae52011f8e257f6b6015308cc352373013020d711fa05049137f257"} Jul 07 19:22:32.276666 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.276422 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:32.278409 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.277944 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" event={"ID":"bbd63a26-061c-4849-936b-c2206893f23a","Type":"ContainerStarted","Data":"2f0f155d17957f744c90d110b3be15d202e35ad3d515b7dd17adbf0d4d85e091"} Jul 07 19:22:32.278621 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.278594 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" Jul 07 19:22:32.295419 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.295359 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d9f88df9f-r7px5" podStartSLOduration=1.504253699 podStartE2EDuration="27.295345606s" totalImagesPullingTime="25.791091907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:32.293720392 +0000 UTC m=+98.195598929" watchObservedRunningTime="2026-07-07 19:22:32.295345606 +0000 UTC m=+98.197224141" Jul 07 19:22:32.458103 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.458071 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:32.563065 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.563019 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:32.827359 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.827323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:32.888063 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.888024 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:32.991401 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:32.991353 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78db467dbd-64sfj"] Jul 07 19:22:33.074609 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.074577 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-88vw9" Jul 07 19:22:33.083444 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.083351 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:33.208862 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.208820 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:33.286196 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.286158 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" event={"ID":"bbd63a26-061c-4849-936b-c2206893f23a","Type":"ContainerStarted","Data":"090c9cbaa813fe35626797d0db3801534d992232da85ce534a4a470cd5a1b072"} Jul 07 19:22:33.286782 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.286208 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" event={"ID":"bbd63a26-061c-4849-936b-c2206893f23a","Type":"ContainerStarted","Data":"47d9413e0cc75e2ae57c3bf782d18580a224f7e26113c8628b959bd8f00a4745"} Jul 07 19:22:33.306951 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.306903 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855c66bc99-k4pdn" podStartSLOduration=2.411932818 podStartE2EDuration="28.306884452s" totalImagesPullingTime="25.894951634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:22:33.305388586 +0000 UTC m=+99.207267113" watchObservedRunningTime="2026-07-07 19:22:33.306884452 +0000 UTC m=+99.208762988" Jul 07 19:22:33.308749 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.308718 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:33.408766 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.408679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:33.845512 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:33.845470 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:35.461665 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:35.461625 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:35.563524 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:35.563486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:35.664606 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:35.664574 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:38.196940 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:38.196903 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:38.301416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:38.301363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:39.347316 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:39.347287 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:41.920479 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:41.920439 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:42.014338 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:42.014303 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:43.507041 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:43.507005 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:43.569799 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:43.569769 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:44.581354 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:44.581323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:44.681450 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:44.681418 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:44.744607 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:44.744581 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:58.016289 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.016245 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerName="registry" containerID="cri-o://d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5" gracePeriod=30 Jul 07 19:22:58.132242 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.132169 2561 patch_prober.go:28] interesting pod/image-registry-78db467dbd-64sfj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body= Jul 07 19:22:58.132242 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.132210 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" Jul 07 19:22:58.260326 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.260304 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:58.382857 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.382769 2561 generic.go:356] "Generic (PLEG): container finished" podID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerID="d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5" exitCode=0 Jul 07 19:22:58.382857 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.382813 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" Jul 07 19:22:58.383042 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.382856 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" event={"ID":"c36715ae-d4d5-4567-b01d-7997b87573ac","Type":"ContainerDied","Data":"d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5"} Jul 07 19:22:58.383042 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.382906 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78db467dbd-64sfj" event={"ID":"c36715ae-d4d5-4567-b01d-7997b87573ac","Type":"ContainerDied","Data":"612efc1d404733b8dbe55b21d69fd88f25d39fbae573eaf52441aa85dbddbe06"} Jul 07 19:22:58.383042 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.382927 2561 scope.go:117] "RemoveContainer" containerID="d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5" Jul 07 19:22:58.385074 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385059 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-bound-sa-token\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385176 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385115 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-certificates\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385287 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-trusted-ca\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385327 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-image-registry-private-configuration\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385366 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxtrz\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-kube-api-access-cxtrz\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385419 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c36715ae-d4d5-4567-b01d-7997b87573ac-ca-trust-extracted\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385451 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385489 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-installation-pull-secrets\") pod \"c36715ae-d4d5-4567-b01d-7997b87573ac\" (UID: \"c36715ae-d4d5-4567-b01d-7997b87573ac\") " Jul 07 19:22:58.385677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385602 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:58.385934 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385706 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:22:58.385934 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.385756 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-certificates\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.388166 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.388141 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:22:58.388296 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.388275 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:22:58.388397 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.388313 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:22:58.388506 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.388479 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:22:58.388560 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.388525 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-kube-api-access-cxtrz" (OuterVolumeSpecName: "kube-api-access-cxtrz") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "kube-api-access-cxtrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:22:58.392054 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.392038 2561 scope.go:117] "RemoveContainer" containerID="d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5" Jul 07 19:22:58.392326 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:22:58.392305 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5\": container with ID starting with d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5 not found: ID does not exist" containerID="d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5" Jul 07 19:22:58.392420 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.392336 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5"} err="failed to get container status \"d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5\": rpc error: code = NotFound desc = could not find container \"d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5\": container with ID starting with d2dfb29c557e3cb74cc5bb24e855552c723034a42a3e5abaa4f019c5299f8bc5 not found: ID does not exist" Jul 07 19:22:58.394313 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.394291 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36715ae-d4d5-4567-b01d-7997b87573ac-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c36715ae-d4d5-4567-b01d-7997b87573ac" (UID: "c36715ae-d4d5-4567-b01d-7997b87573ac"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:22:58.486290 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486259 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c36715ae-d4d5-4567-b01d-7997b87573ac-trusted-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.486290 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486290 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-image-registry-private-configuration\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.486495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486302 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxtrz\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-kube-api-access-cxtrz\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.486495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486312 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c36715ae-d4d5-4567-b01d-7997b87573ac-ca-trust-extracted\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.486495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486320 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-registry-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.486495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486329 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c36715ae-d4d5-4567-b01d-7997b87573ac-installation-pull-secrets\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.486495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.486337 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c36715ae-d4d5-4567-b01d-7997b87573ac-bound-sa-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:22:58.589899 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.589864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:22:58.708798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.708775 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78db467dbd-64sfj"] Jul 07 19:22:58.711488 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.711466 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78db467dbd-64sfj"] Jul 07 19:22:58.739512 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:22:58.739480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:00.392702 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:00.392673 2561 generic.go:356] "Generic (PLEG): container finished" podID="6b6f199d-ac3f-445b-8458-bbba5009bd0c" containerID="aa83ab0db5b11c1d8840484ca6af4fd234bd3c94936e02fd315f2c045a0f3a7e" exitCode=0 Jul 07 19:23:00.393021 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:00.392748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" event={"ID":"6b6f199d-ac3f-445b-8458-bbba5009bd0c","Type":"ContainerDied","Data":"aa83ab0db5b11c1d8840484ca6af4fd234bd3c94936e02fd315f2c045a0f3a7e"} Jul 07 19:23:00.393125 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:00.393109 2561 scope.go:117] "RemoveContainer" containerID="aa83ab0db5b11c1d8840484ca6af4fd234bd3c94936e02fd315f2c045a0f3a7e" Jul 07 19:23:00.622653 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:00.622583 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" path="/var/lib/kubelet/pods/c36715ae-d4d5-4567-b01d-7997b87573ac/volumes" Jul 07 19:23:01.397610 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:01.397574 2561 generic.go:356] "Generic (PLEG): container finished" podID="fb275177-b68f-440a-b440-9fa8d5f4c2c3" containerID="ff046c301b22f847995a89e816ad06148db923ffb92a56ba161187490a2c56e3" exitCode=0 Jul 07 19:23:01.398008 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:01.397654 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" event={"ID":"fb275177-b68f-440a-b440-9fa8d5f4c2c3","Type":"ContainerDied","Data":"ff046c301b22f847995a89e816ad06148db923ffb92a56ba161187490a2c56e3"} Jul 07 19:23:01.398109 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:01.398094 2561 scope.go:117] "RemoveContainer" containerID="ff046c301b22f847995a89e816ad06148db923ffb92a56ba161187490a2c56e3" Jul 07 19:23:01.399505 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:01.399484 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-r7tl9" event={"ID":"6b6f199d-ac3f-445b-8458-bbba5009bd0c","Type":"ContainerStarted","Data":"7217dc2f77786495a1d22880f07fbd5ed6dd2aa9281fbc747c9300c3874dc614"} Jul 07 19:23:02.405176 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:02.405145 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7c47h" event={"ID":"fb275177-b68f-440a-b440-9fa8d5f4c2c3","Type":"ContainerStarted","Data":"29d2114808cfeb18c74d86d0a22979ba865fbb2b917c864b70bb73dc409efd5b"} Jul 07 19:23:06.421390 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:06.421342 2561 generic.go:356] "Generic (PLEG): container finished" podID="8f867d34-c4ae-44fb-a644-3faa3118dc5d" containerID="5bc3965a5f489bd6e824c1fdd43abd7ea536dc1a9db0845b124e24c75ac6d9f0" exitCode=0 Jul 07 19:23:06.421793 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:06.421418 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" event={"ID":"8f867d34-c4ae-44fb-a644-3faa3118dc5d","Type":"ContainerDied","Data":"5bc3965a5f489bd6e824c1fdd43abd7ea536dc1a9db0845b124e24c75ac6d9f0"} Jul 07 19:23:06.421793 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:06.421724 2561 scope.go:117] "RemoveContainer" containerID="5bc3965a5f489bd6e824c1fdd43abd7ea536dc1a9db0845b124e24c75ac6d9f0" Jul 07 19:23:07.427579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:07.427549 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-jdkm8" event={"ID":"8f867d34-c4ae-44fb-a644-3faa3118dc5d","Type":"ContainerStarted","Data":"738e33f9c0d98c6913bd0e950758c99697ac5e8e1ce6a21328c6ec442d855650"} Jul 07 19:23:13.518019 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:13.517975 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:13.583632 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:13.583601 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:16.790067 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:16.790029 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:16.890888 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:16.890848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:39.250356 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:39.250323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:39.319833 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:39.319802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:51.247051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:51.246968 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:51.353345 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:51.353312 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:58.534703 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:58.534668 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:58.633486 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:58.633452 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:58.794083 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:58.794002 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:23:58.857697 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:23:58.857651 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:09.958480 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:09.958431 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:10.072617 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:10.072583 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:32.395106 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:32.395065 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:32.493326 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:32.493291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:39.878789 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:39.878747 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:39.972705 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:39.972666 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:52.408970 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:52.408934 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:24:52.477785 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:24:52.477750 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:01.452640 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:01.452608 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:02.512854 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:02.512823 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:05.082747 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:05.082710 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:05.159848 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:05.159807 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:09.901135 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:09.901096 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:09.999633 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:09.999593 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:10.351166 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:10.351138 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:10.513953 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:10.513920 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:11.943525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:11.943486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:12.320504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:12.320417 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:14.346747 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:14.346712 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:20.730619 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:20.730533 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:22.150269 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:22.150240 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:22.251433 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:22.251407 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:28.436505 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:28.436468 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:28.753181 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:28.753092 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:28.855049 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:28.855015 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:30.815336 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:30.815297 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:36.589813 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:36.589779 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:36.689947 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:36.689920 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:44.563942 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:44.563910 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:44.662292 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:44.662258 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:46.713580 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:46.713533 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:46.855742 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:46.855711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:47.388937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.388903 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:47.659571 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.659503 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:47.767856 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.767829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:47.837888 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.837863 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7"] Jul 07 19:25:47.838314 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.838293 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerName="registry" Jul 07 19:25:47.838314 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.838317 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerName="registry" Jul 07 19:25:47.838519 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.838431 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36715ae-d4d5-4567-b01d-7997b87573ac" containerName="registry" Jul 07 19:25:47.844345 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.842605 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:47.847900 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.847880 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 19:25:47.847900 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.847894 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 19:25:47.848264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.848247 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 19:25:47.848507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.848494 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-smps6\"" Jul 07 19:25:47.848996 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.848983 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 19:25:47.968618 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.968598 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68a70b92-cb6b-4112-9bef-1700b07dd50e-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:47.968729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.968652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68a70b92-cb6b-4112-9bef-1700b07dd50e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:47.968729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.968673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a70b92-cb6b-4112-9bef-1700b07dd50e-tmp\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:47.968729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.968699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jzl9\" (UniqueName: \"kubernetes.io/projected/68a70b92-cb6b-4112-9bef-1700b07dd50e-kube-api-access-5jzl9\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:47.968877 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.968770 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68a70b92-cb6b-4112-9bef-1700b07dd50e-manifests\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:47.972009 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:47.971989 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7"] Jul 07 19:25:48.004983 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.004960 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:48.069886 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.069863 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68a70b92-cb6b-4112-9bef-1700b07dd50e-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.069979 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.069912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68a70b92-cb6b-4112-9bef-1700b07dd50e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.069979 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.069929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a70b92-cb6b-4112-9bef-1700b07dd50e-tmp\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.069979 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.069955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jzl9\" (UniqueName: \"kubernetes.io/projected/68a70b92-cb6b-4112-9bef-1700b07dd50e-kube-api-access-5jzl9\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.070098 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.069982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68a70b92-cb6b-4112-9bef-1700b07dd50e-manifests\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.070422 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.070371 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68a70b92-cb6b-4112-9bef-1700b07dd50e-manifests\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.072270 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.072247 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a70b92-cb6b-4112-9bef-1700b07dd50e-tmp\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.072364 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.072346 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68a70b92-cb6b-4112-9bef-1700b07dd50e-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.072525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.072507 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68a70b92-cb6b-4112-9bef-1700b07dd50e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.079963 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.079943 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jzl9\" (UniqueName: \"kubernetes.io/projected/68a70b92-cb6b-4112-9bef-1700b07dd50e-kube-api-access-5jzl9\") pod \"opendatahub-operator-controller-manager-79b594cbdb-kg2c7\" (UID: \"68a70b92-cb6b-4112-9bef-1700b07dd50e\") " pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.108941 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.108915 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:48.153287 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.153270 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:48.212539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.212509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:48.294287 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.294254 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7"] Jul 07 19:25:48.294468 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.294410 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:48.298082 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:25:48.298057 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a70b92_cb6b_4112_9bef_1700b07dd50e.slice/crio-715fd53b75fc8901aad9de7cff970cbe4312ff59ab232e981846bf911440819f WatchSource:0}: Error finding container 715fd53b75fc8901aad9de7cff970cbe4312ff59ab232e981846bf911440819f: Status 404 returned error can't find the container with id 715fd53b75fc8901aad9de7cff970cbe4312ff59ab232e981846bf911440819f Jul 07 19:25:48.361193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.361165 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:48.459314 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:48.459279 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:49.038332 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:49.038297 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" event={"ID":"68a70b92-cb6b-4112-9bef-1700b07dd50e","Type":"ContainerStarted","Data":"715fd53b75fc8901aad9de7cff970cbe4312ff59ab232e981846bf911440819f"} Jul 07 19:25:52.054443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:52.054412 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" event={"ID":"68a70b92-cb6b-4112-9bef-1700b07dd50e","Type":"ContainerStarted","Data":"6160c6bcabea915fbea59f5b170c853fd002c1287b2a9d535f4d2f00ff29c6a5"} Jul 07 19:25:53.060097 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:53.060067 2561 generic.go:356] "Generic (PLEG): container finished" podID="68a70b92-cb6b-4112-9bef-1700b07dd50e" containerID="6160c6bcabea915fbea59f5b170c853fd002c1287b2a9d535f4d2f00ff29c6a5" exitCode=0 Jul 07 19:25:53.060488 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:53.060108 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" event={"ID":"68a70b92-cb6b-4112-9bef-1700b07dd50e","Type":"ContainerDied","Data":"6160c6bcabea915fbea59f5b170c853fd002c1287b2a9d535f4d2f00ff29c6a5"} Jul 07 19:25:54.065855 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:54.065820 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" event={"ID":"68a70b92-cb6b-4112-9bef-1700b07dd50e","Type":"ContainerStarted","Data":"5cc94c5fff79dbec6468a7684d3a1499c4630bce947529dd39f994cbfa105c1b"} Jul 07 19:25:54.066257 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:54.065979 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:25:54.093506 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:54.093455 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" podStartSLOduration=2.603699017 podStartE2EDuration="7.093442797s" totalImagesPullingTime="3.477841416s" totalInitContainerRuntime="1.011902364s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:25:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:54.092368516 +0000 UTC m=+299.994247050" watchObservedRunningTime="2026-07-07 19:25:54.093442797 +0000 UTC m=+299.995321331" Jul 07 19:25:54.496036 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:54.496014 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:25:54.499300 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:54.499271 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:25:54.507860 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:54.507841 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 19:25:55.106775 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:55.106742 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:56.576495 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:56.576460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:59.768902 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:59.768863 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:25:59.869917 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:25:59.869887 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:04.903233 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:04.903196 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:05.002517 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:05.002482 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:05.071945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:05.071923 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79b594cbdb-kg2c7" Jul 07 19:26:05.340095 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:05.340059 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:05.438663 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:05.438626 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:07.007591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:07.007556 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:07.170830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:07.170790 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:07.758077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:07.758040 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:07.857990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:07.857959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:07.960327 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:07.960291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:08.386113 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:08.386076 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:08.597773 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:08.597720 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:18.375821 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.375783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:18.418338 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.418308 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65ff665485-r72v9"] Jul 07 19:26:18.420802 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.420787 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.431880 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.431860 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 19:26:18.431974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.431926 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 19:26:18.438944 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.438919 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65ff665485-r72v9"] Jul 07 19:26:18.482870 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.482848 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fab8db2-af52-4a28-a4ad-3fa40b8570da-tmp\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.482990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.482897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fab8db2-af52-4a28-a4ad-3fa40b8570da-tls-certs\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.482990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.482930 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx2w8\" (UniqueName: \"kubernetes.io/projected/3fab8db2-af52-4a28-a4ad-3fa40b8570da-kube-api-access-qx2w8\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.485995 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.485959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:18.584429 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.584362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fab8db2-af52-4a28-a4ad-3fa40b8570da-tmp\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.584613 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.584504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fab8db2-af52-4a28-a4ad-3fa40b8570da-tls-certs\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.584613 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.584557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx2w8\" (UniqueName: \"kubernetes.io/projected/3fab8db2-af52-4a28-a4ad-3fa40b8570da-kube-api-access-qx2w8\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.586758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.586725 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fab8db2-af52-4a28-a4ad-3fa40b8570da-tmp\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.587080 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.587056 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fab8db2-af52-4a28-a4ad-3fa40b8570da-tls-certs\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.605559 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.605536 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx2w8\" (UniqueName: \"kubernetes.io/projected/3fab8db2-af52-4a28-a4ad-3fa40b8570da-kube-api-access-qx2w8\") pod \"kube-auth-proxy-65ff665485-r72v9\" (UID: \"3fab8db2-af52-4a28-a4ad-3fa40b8570da\") " pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.621635 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.621597 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:18.673054 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.672981 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8"] Jul 07 19:26:18.676099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.676083 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:18.678848 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.678826 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 19:26:18.678955 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.678857 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wkdz9\"" Jul 07 19:26:18.690835 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.690813 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9"] Jul 07 19:26:18.693244 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.693225 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8"] Jul 07 19:26:18.693339 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.693302 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:18.696657 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.696634 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 19:26:18.697611 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.697591 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2mtk7\"" Jul 07 19:26:18.714637 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.714614 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9"] Jul 07 19:26:18.725045 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.725020 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:18.729644 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.729627 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" Jul 07 19:26:18.786336 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.786311 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a309fe58-09b9-4a26-96c8-882c6dbea038-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:18.786517 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.786475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqqcj\" (UniqueName: \"kubernetes.io/projected/86add30f-218e-45ae-a197-a2f50df4c7ea-kube-api-access-sqqcj\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:18.786606 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.786556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94c69\" (UniqueName: \"kubernetes.io/projected/a309fe58-09b9-4a26-96c8-882c6dbea038-kube-api-access-94c69\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:18.786667 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.786619 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:18.879464 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.879433 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65ff665485-r72v9"] Jul 07 19:26:18.880298 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:26:18.880269 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fab8db2_af52_4a28_a4ad_3fa40b8570da.slice/crio-c679d4991dd8a6e8170468f1c4d9a7c0d9164e319fbd2709d52e36a2e3d6eb15 WatchSource:0}: Error finding container c679d4991dd8a6e8170468f1c4d9a7c0d9164e319fbd2709d52e36a2e3d6eb15: Status 404 returned error can't find the container with id c679d4991dd8a6e8170468f1c4d9a7c0d9164e319fbd2709d52e36a2e3d6eb15 Jul 07 19:26:18.882464 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.882448 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:26:18.887296 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.887274 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:18.887400 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.887343 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a309fe58-09b9-4a26-96c8-882c6dbea038-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:18.887468 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:26:18.887440 2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 07 19:26:18.887468 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:26:18.887449 2561 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 19:26:18.887563 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.887461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqqcj\" (UniqueName: \"kubernetes.io/projected/86add30f-218e-45ae-a197-a2f50df4c7ea-kube-api-access-sqqcj\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:18.887563 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:26:18.887496 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert podName:86add30f-218e-45ae-a197-a2f50df4c7ea nodeName:}" failed. No retries permitted until 2026-07-07 19:26:19.387478747 +0000 UTC m=+325.289357264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert") pod "kserve-controller-manager-d6f4b65b8-7cgz8" (UID: "86add30f-218e-45ae-a197-a2f50df4c7ea") : secret "kserve-webhook-server-cert" not found Jul 07 19:26:18.887563 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:26:18.887509 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a309fe58-09b9-4a26-96c8-882c6dbea038-cert podName:a309fe58-09b9-4a26-96c8-882c6dbea038 nodeName:}" failed. No retries permitted until 2026-07-07 19:26:19.387504164 +0000 UTC m=+325.289382683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a309fe58-09b9-4a26-96c8-882c6dbea038-cert") pod "llmisvc-controller-manager-777c6bd7d5-xmvd9" (UID: "a309fe58-09b9-4a26-96c8-882c6dbea038") : secret "llmisvc-webhook-server-cert" not found Jul 07 19:26:18.887563 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.887525 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94c69\" (UniqueName: \"kubernetes.io/projected/a309fe58-09b9-4a26-96c8-882c6dbea038-kube-api-access-94c69\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:18.900042 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.900013 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94c69\" (UniqueName: \"kubernetes.io/projected/a309fe58-09b9-4a26-96c8-882c6dbea038-kube-api-access-94c69\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:18.904765 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:18.904744 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqqcj\" (UniqueName: \"kubernetes.io/projected/86add30f-218e-45ae-a197-a2f50df4c7ea-kube-api-access-sqqcj\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:19.168226 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.168184 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" event={"ID":"3fab8db2-af52-4a28-a4ad-3fa40b8570da","Type":"ContainerStarted","Data":"c679d4991dd8a6e8170468f1c4d9a7c0d9164e319fbd2709d52e36a2e3d6eb15"} Jul 07 19:26:19.360137 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.360094 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:19.390825 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.390799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:19.391209 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.390854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a309fe58-09b9-4a26-96c8-882c6dbea038-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:19.391209 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:26:19.390947 2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 07 19:26:19.391209 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:26:19.391008 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert podName:86add30f-218e-45ae-a197-a2f50df4c7ea nodeName:}" failed. No retries permitted until 2026-07-07 19:26:20.390991391 +0000 UTC m=+326.292869908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert") pod "kserve-controller-manager-d6f4b65b8-7cgz8" (UID: "86add30f-218e-45ae-a197-a2f50df4c7ea") : secret "kserve-webhook-server-cert" not found Jul 07 19:26:19.393179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.393162 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a309fe58-09b9-4a26-96c8-882c6dbea038-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xmvd9\" (UID: \"a309fe58-09b9-4a26-96c8-882c6dbea038\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:19.507090 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.507061 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:19.603455 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.603432 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:19.744406 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.744320 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9"] Jul 07 19:26:19.746605 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:26:19.746569 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda309fe58_09b9_4a26_96c8_882c6dbea038.slice/crio-0c5491f97f78f84d10f8295825a5d3085cf2d3d4a71ce7c33ffc98441233f012 WatchSource:0}: Error finding container 0c5491f97f78f84d10f8295825a5d3085cf2d3d4a71ce7c33ffc98441233f012: Status 404 returned error can't find the container with id 0c5491f97f78f84d10f8295825a5d3085cf2d3d4a71ce7c33ffc98441233f012 Jul 07 19:26:19.900544 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:19.900467 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:20.006736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.006705 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:20.174093 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.174012 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" event={"ID":"a309fe58-09b9-4a26-96c8-882c6dbea038","Type":"ContainerStarted","Data":"0c5491f97f78f84d10f8295825a5d3085cf2d3d4a71ce7c33ffc98441233f012"} Jul 07 19:26:20.204396 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.204342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:20.400966 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.400935 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:20.403144 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.403121 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86add30f-218e-45ae-a197-a2f50df4c7ea-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cgz8\" (UID: \"86add30f-218e-45ae-a197-a2f50df4c7ea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:20.486430 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.486369 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:20.732308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.732270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:20.962179 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:20.962093 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:21.030030 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:21.029973 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8"] Jul 07 19:26:21.090243 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:26:21.090209 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86add30f_218e_45ae_a197_a2f50df4c7ea.slice/crio-8b113569d04bb1d3f89ccbb391aec51055441d111b2a9226bab6aa292fba035d WatchSource:0}: Error finding container 8b113569d04bb1d3f89ccbb391aec51055441d111b2a9226bab6aa292fba035d: Status 404 returned error can't find the container with id 8b113569d04bb1d3f89ccbb391aec51055441d111b2a9226bab6aa292fba035d Jul 07 19:26:21.184691 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:21.184652 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" event={"ID":"86add30f-218e-45ae-a197-a2f50df4c7ea","Type":"ContainerStarted","Data":"8b113569d04bb1d3f89ccbb391aec51055441d111b2a9226bab6aa292fba035d"} Jul 07 19:26:22.195650 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:22.195611 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" event={"ID":"3fab8db2-af52-4a28-a4ad-3fa40b8570da","Type":"ContainerStarted","Data":"2e08297cb545033fe0204f4fdfd94df64792017d2431c0e796840797f275debf"} Jul 07 19:26:22.231581 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:22.231538 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65ff665485-r72v9" podStartSLOduration=1.046962075 podStartE2EDuration="4.231523819s" totalImagesPullingTime="3.184561744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:22.230410241 +0000 UTC m=+328.132288795" watchObservedRunningTime="2026-07-07 19:26:22.231523819 +0000 UTC m=+328.133402355" Jul 07 19:26:22.806632 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:22.806589 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:23.201397 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:23.201267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" event={"ID":"a309fe58-09b9-4a26-96c8-882c6dbea038","Type":"ContainerStarted","Data":"44a8d56cceed7535bbf6a737bd0a4cc6a6491d6a87c029974874481a116e0dcb"} Jul 07 19:26:23.225401 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:23.225331 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" podStartSLOduration=2.123522287 podStartE2EDuration="5.225314326s" totalImagesPullingTime="3.101792039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:23.224921742 +0000 UTC m=+329.126800276" watchObservedRunningTime="2026-07-07 19:26:23.225314326 +0000 UTC m=+329.127192865" Jul 07 19:26:24.030932 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:24.030892 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:24.146317 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:24.146277 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:24.207754 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:24.207716 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:24.490156 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:24.490120 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:24.588741 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:24.588713 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:25.213906 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.213869 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" event={"ID":"86add30f-218e-45ae-a197-a2f50df4c7ea","Type":"ContainerStarted","Data":"2f187d0bf60a2d75c9988d12839efd4522617919b3662bd8693cec029dcce97b"} Jul 07 19:26:25.241921 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.241879 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" podStartSLOduration=3.991920679 podStartE2EDuration="7.241865657s" totalImagesPullingTime="3.249944978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:25.241678296 +0000 UTC m=+331.143556849" watchObservedRunningTime="2026-07-07 19:26:25.241865657 +0000 UTC m=+331.143744191" Jul 07 19:26:25.520691 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.520602 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc"] Jul 07 19:26:25.523317 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.523297 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.526022 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.525994 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vfm78\"" Jul 07 19:26:25.526117 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.525998 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 19:26:25.539268 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.539245 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc"] Jul 07 19:26:25.559182 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.559144 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:25.648525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76980d86-780f-4bd2-a924-e588fe09c019-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648666 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648532 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648666 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648567 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8shqg\" (UniqueName: \"kubernetes.io/projected/76980d86-780f-4bd2-a924-e588fe09c019-kube-api-access-8shqg\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648666 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648638 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648740 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648778 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76980d86-780f-4bd2-a924-e588fe09c019-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.648824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.648805 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76980d86-780f-4bd2-a924-e588fe09c019-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750071 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750042 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750191 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750191 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750097 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750309 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750309 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76980d86-780f-4bd2-a924-e588fe09c019-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750439 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76980d86-780f-4bd2-a924-e588fe09c019-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750439 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76980d86-780f-4bd2-a924-e588fe09c019-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750439 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750428 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750481 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750534 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750817 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8shqg\" (UniqueName: \"kubernetes.io/projected/76980d86-780f-4bd2-a924-e588fe09c019-kube-api-access-8shqg\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.750879 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.750813 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.751122 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.751101 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76980d86-780f-4bd2-a924-e588fe09c019-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.752486 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.752464 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76980d86-780f-4bd2-a924-e588fe09c019-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.753196 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.753179 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76980d86-780f-4bd2-a924-e588fe09c019-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.767207 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.767187 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8shqg\" (UniqueName: \"kubernetes.io/projected/76980d86-780f-4bd2-a924-e588fe09c019-kube-api-access-8shqg\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.767450 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.767430 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76980d86-780f-4bd2-a924-e588fe09c019-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5xvc\" (UID: \"76980d86-780f-4bd2-a924-e588fe09c019\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.802642 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.802581 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:25.833919 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.833896 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:25.923701 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.923648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:25.970239 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:25.970217 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc"] Jul 07 19:26:25.971568 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:26:25.971540 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76980d86_780f_4bd2_a924_e588fe09c019.slice/crio-20d294f104dc1c40023093463891925831e832031e8781083f208bf3e8264c75 WatchSource:0}: Error finding container 20d294f104dc1c40023093463891925831e832031e8781083f208bf3e8264c75: Status 404 returned error can't find the container with id 20d294f104dc1c40023093463891925831e832031e8781083f208bf3e8264c75 Jul 07 19:26:26.220103 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:26.220069 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" event={"ID":"76980d86-780f-4bd2-a924-e588fe09c019","Type":"ContainerStarted","Data":"20d294f104dc1c40023093463891925831e832031e8781083f208bf3e8264c75"} Jul 07 19:26:26.220544 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:26.220227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:26:28.241199 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:28.241166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:26:28.511208 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:28.511156 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:26:28.511279 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:28.511255 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:26:28.511320 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:28.511282 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:26:29.237221 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:29.237183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" event={"ID":"76980d86-780f-4bd2-a924-e588fe09c019","Type":"ContainerStarted","Data":"199cdfb1d2629ae43f706c35731398a98878a0114e95382a78c9920b8e7f25df"} Jul 07 19:26:29.259700 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:29.259653 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" podStartSLOduration=1.722298283 podStartE2EDuration="4.259637096s" totalImagesPullingTime="2.537338813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:29.257208217 +0000 UTC m=+335.159086771" watchObservedRunningTime="2026-07-07 19:26:29.259637096 +0000 UTC m=+335.161515631" Jul 07 19:26:29.834591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:29.834556 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:29.838788 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:29.838763 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:30.242172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:30.242145 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:30.243149 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:30.243132 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5xvc" Jul 07 19:26:55.216124 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:55.216048 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xmvd9" Jul 07 19:26:57.231228 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:26:57.231198 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cgz8" Jul 07 19:27:08.570043 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.570007 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:08.661448 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.661421 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:08.711559 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.711533 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vf4z4"] Jul 07 19:27:08.714515 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.714493 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:08.717435 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.717358 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 19:27:08.717435 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.717359 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pq7q8\"" Jul 07 19:27:08.718054 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.718032 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 19:27:08.725370 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.725346 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vf4z4"] Jul 07 19:27:08.764969 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.764941 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:08.883623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.883553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzfq7\" (UniqueName: \"kubernetes.io/projected/c8049a27-c6dc-4f58-a036-967438714cc0-kube-api-access-nzfq7\") pod \"kuadrant-operator-catalog-vf4z4\" (UID: \"c8049a27-c6dc-4f58-a036-967438714cc0\") " pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:08.984089 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:08.984065 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzfq7\" (UniqueName: \"kubernetes.io/projected/c8049a27-c6dc-4f58-a036-967438714cc0-kube-api-access-nzfq7\") pod \"kuadrant-operator-catalog-vf4z4\" (UID: \"c8049a27-c6dc-4f58-a036-967438714cc0\") " pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:09.003849 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.003831 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzfq7\" (UniqueName: \"kubernetes.io/projected/c8049a27-c6dc-4f58-a036-967438714cc0-kube-api-access-nzfq7\") pod \"kuadrant-operator-catalog-vf4z4\" (UID: \"c8049a27-c6dc-4f58-a036-967438714cc0\") " pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:09.027750 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.027727 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:09.071466 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.071438 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vf4z4"] Jul 07 19:27:09.158500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.158474 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vf4z4"] Jul 07 19:27:09.161101 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:27:09.161075 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8049a27_c6dc_4f58_a036_967438714cc0.slice/crio-bcd7b1d0fe973d9454674867ce7f5bc7bfcedb2e64f49a9aa3671c8fd2f097e0 WatchSource:0}: Error finding container bcd7b1d0fe973d9454674867ce7f5bc7bfcedb2e64f49a9aa3671c8fd2f097e0: Status 404 returned error can't find the container with id bcd7b1d0fe973d9454674867ce7f5bc7bfcedb2e64f49a9aa3671c8fd2f097e0 Jul 07 19:27:09.282923 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.282895 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6wz2b"] Jul 07 19:27:09.285912 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.285897 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:09.296782 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.296759 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6wz2b"] Jul 07 19:27:09.386678 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.386651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgk4d\" (UniqueName: \"kubernetes.io/projected/51115018-6adb-4301-b7c7-c157c5a2172d-kube-api-access-jgk4d\") pod \"kuadrant-operator-catalog-6wz2b\" (UID: \"51115018-6adb-4301-b7c7-c157c5a2172d\") " pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:09.405619 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.405597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" event={"ID":"c8049a27-c6dc-4f58-a036-967438714cc0","Type":"ContainerStarted","Data":"bcd7b1d0fe973d9454674867ce7f5bc7bfcedb2e64f49a9aa3671c8fd2f097e0"} Jul 07 19:27:09.487993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.487970 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgk4d\" (UniqueName: \"kubernetes.io/projected/51115018-6adb-4301-b7c7-c157c5a2172d-kube-api-access-jgk4d\") pod \"kuadrant-operator-catalog-6wz2b\" (UID: \"51115018-6adb-4301-b7c7-c157c5a2172d\") " pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:09.497030 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.497014 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgk4d\" (UniqueName: \"kubernetes.io/projected/51115018-6adb-4301-b7c7-c157c5a2172d-kube-api-access-jgk4d\") pod \"kuadrant-operator-catalog-6wz2b\" (UID: \"51115018-6adb-4301-b7c7-c157c5a2172d\") " pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:09.511993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.511966 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:09.595190 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.595163 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:09.722909 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.722866 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:09.733704 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:09.733615 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6wz2b"] Jul 07 19:27:09.770585 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:27:09.770520 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51115018_6adb_4301_b7c7_c157c5a2172d.slice/crio-073488118e1637f0dca5502421b83b3c68be352adadaf6b8c344c5a41dccec79 WatchSource:0}: Error finding container 073488118e1637f0dca5502421b83b3c68be352adadaf6b8c344c5a41dccec79: Status 404 returned error can't find the container with id 073488118e1637f0dca5502421b83b3c68be352adadaf6b8c344c5a41dccec79 Jul 07 19:27:10.413748 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:10.413706 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" event={"ID":"51115018-6adb-4301-b7c7-c157c5a2172d","Type":"ContainerStarted","Data":"073488118e1637f0dca5502421b83b3c68be352adadaf6b8c344c5a41dccec79"} Jul 07 19:27:10.932998 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:27:10.932956 2561 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:dd64bf2dd177757451a98fcdc999a339c35dee5d9872d8f4dc69c8f3c4dd0112: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/kuadrant/kuadrant-operator-catalog:v1.4.2" Jul 07 19:27:10.933479 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:27:10.933091 2561 kuberuntime_manager.go:1459] "Unhandled Error" err="container registry-server start failed in pod kuadrant-operator-catalog-vf4z4_kuadrant-system(c8049a27-c6dc-4f58-a036-967438714cc0): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:dd64bf2dd177757451a98fcdc999a339c35dee5d9872d8f4dc69c8f3c4dd0112: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 07 19:27:10.933479 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:27:10.933132 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry-server\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:dd64bf2dd177757451a98fcdc999a339c35dee5d9872d8f4dc69c8f3c4dd0112: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" podUID="c8049a27-c6dc-4f58-a036-967438714cc0" Jul 07 19:27:11.419456 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:11.419417 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" event={"ID":"51115018-6adb-4301-b7c7-c157c5a2172d","Type":"ContainerStarted","Data":"b582afc1e4867f81030c908f4a79f64057148c588fce3767d9797ae5abb42877"} Jul 07 19:27:11.444137 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:11.444024 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" podStartSLOduration=1.020401968 podStartE2EDuration="2.444007803s" totalImagesPullingTime="1.423605835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:27:11.442236763 +0000 UTC m=+377.344115323" watchObservedRunningTime="2026-07-07 19:27:11.444007803 +0000 UTC m=+377.345886329" Jul 07 19:27:11.541219 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:11.541196 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:11.707667 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:11.707594 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzfq7\" (UniqueName: \"kubernetes.io/projected/c8049a27-c6dc-4f58-a036-967438714cc0-kube-api-access-nzfq7\") pod \"c8049a27-c6dc-4f58-a036-967438714cc0\" (UID: \"c8049a27-c6dc-4f58-a036-967438714cc0\") " Jul 07 19:27:11.709634 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:11.709609 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8049a27-c6dc-4f58-a036-967438714cc0-kube-api-access-nzfq7" (OuterVolumeSpecName: "kube-api-access-nzfq7") pod "c8049a27-c6dc-4f58-a036-967438714cc0" (UID: "c8049a27-c6dc-4f58-a036-967438714cc0"). InnerVolumeSpecName "kube-api-access-nzfq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:27:11.808000 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:11.807975 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzfq7\" (UniqueName: \"kubernetes.io/projected/c8049a27-c6dc-4f58-a036-967438714cc0-kube-api-access-nzfq7\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:27:12.424863 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:12.424832 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" Jul 07 19:27:12.425262 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:12.424877 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vf4z4" event={"ID":"c8049a27-c6dc-4f58-a036-967438714cc0","Type":"ContainerDied","Data":"bcd7b1d0fe973d9454674867ce7f5bc7bfcedb2e64f49a9aa3671c8fd2f097e0"} Jul 07 19:27:12.463424 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:12.463402 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vf4z4"] Jul 07 19:27:12.468391 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:12.468360 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vf4z4"] Jul 07 19:27:12.622741 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:12.622713 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8049a27-c6dc-4f58-a036-967438714cc0" path="/var/lib/kubelet/pods/c8049a27-c6dc-4f58-a036-967438714cc0/volumes" Jul 07 19:27:19.595886 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:19.595854 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:19.596285 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:19.595911 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:19.619027 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:19.619004 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:20.479134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:20.479106 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6wz2b" Jul 07 19:27:35.230912 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:35.230870 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:35.296697 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:35.296657 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:38.526811 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:38.526771 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:41.180197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.180160 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k"] Jul 07 19:27:41.182765 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.182749 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:41.185372 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.185348 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r8nml\"" Jul 07 19:27:41.185618 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.185604 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 19:27:41.195561 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.195537 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k"] Jul 07 19:27:41.238372 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.238342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwvwg\" (UniqueName: \"kubernetes.io/projected/14997e14-d133-408b-bc06-9afbb75e84c6-kube-api-access-nwvwg\") pod \"dns-operator-controller-manager-864dd78c4c-2tm9k\" (UID: \"14997e14-d133-408b-bc06-9afbb75e84c6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:41.339167 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.339132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwvwg\" (UniqueName: \"kubernetes.io/projected/14997e14-d133-408b-bc06-9afbb75e84c6-kube-api-access-nwvwg\") pod \"dns-operator-controller-manager-864dd78c4c-2tm9k\" (UID: \"14997e14-d133-408b-bc06-9afbb75e84c6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:41.355178 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.355145 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwvwg\" (UniqueName: \"kubernetes.io/projected/14997e14-d133-408b-bc06-9afbb75e84c6-kube-api-access-nwvwg\") pod \"dns-operator-controller-manager-864dd78c4c-2tm9k\" (UID: \"14997e14-d133-408b-bc06-9afbb75e84c6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:41.492663 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.492629 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:41.632939 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:41.632914 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k"] Jul 07 19:27:41.635420 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:27:41.635392 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14997e14_d133_408b_bc06_9afbb75e84c6.slice/crio-7f2b2c2771a24e1485cdfd2aa9de359a2452644caa158ebde57d37143f2a1d15 WatchSource:0}: Error finding container 7f2b2c2771a24e1485cdfd2aa9de359a2452644caa158ebde57d37143f2a1d15: Status 404 returned error can't find the container with id 7f2b2c2771a24e1485cdfd2aa9de359a2452644caa158ebde57d37143f2a1d15 Jul 07 19:27:42.126485 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:42.126450 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:42.555519 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:42.555490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" event={"ID":"14997e14-d133-408b-bc06-9afbb75e84c6","Type":"ContainerStarted","Data":"7f2b2c2771a24e1485cdfd2aa9de359a2452644caa158ebde57d37143f2a1d15"} Jul 07 19:27:43.562942 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:43.562916 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" event={"ID":"14997e14-d133-408b-bc06-9afbb75e84c6","Type":"ContainerStarted","Data":"125ffe0803877a69ebc7ed8916dddc9aee212bf1bf09d9464b2f81a5315d2a02"} Jul 07 19:27:43.563246 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:43.562995 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:43.584456 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:43.584415 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" podStartSLOduration=0.758098815 podStartE2EDuration="2.584400021s" totalImagesPullingTime="1.826301206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:27:43.582525167 +0000 UTC m=+409.484403702" watchObservedRunningTime="2026-07-07 19:27:43.584400021 +0000 UTC m=+409.486278555" Jul 07 19:27:45.153128 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:45.153094 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:46.561158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.561123 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w"] Jul 07 19:27:46.566265 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.566249 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.570439 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:27:46.570409 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.572595 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:27:46.572563 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kuadrant-operator-controller-manager-dockercfg-g5xpr\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g5xpr\"" type="*v1.Secret" Jul 07 19:27:46.589267 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.589241 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w"] Jul 07 19:27:46.652977 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.652940 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:46.684243 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.684215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da67d31c-c3df-4723-b097-039e1a52c6e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.684361 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.684250 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc72m\" (UniqueName: \"kubernetes.io/projected/da67d31c-c3df-4723-b097-039e1a52c6e1-kube-api-access-wc72m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.784898 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.784867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da67d31c-c3df-4723-b097-039e1a52c6e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.785035 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.784903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc72m\" (UniqueName: \"kubernetes.io/projected/da67d31c-c3df-4723-b097-039e1a52c6e1-kube-api-access-wc72m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.785243 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.785223 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da67d31c-c3df-4723-b097-039e1a52c6e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:46.820332 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:46.820266 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc72m\" (UniqueName: \"kubernetes.io/projected/da67d31c-c3df-4723-b097-039e1a52c6e1-kube-api-access-wc72m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:47.013930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:47.013897 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:47.526285 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:47.526250 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:47.551713 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:47.551694 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g5xpr\"" Jul 07 19:27:47.558063 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:47.558047 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:47.698275 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:47.698224 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w"] Jul 07 19:27:47.700430 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:27:47.700405 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda67d31c_c3df_4723_b097_039e1a52c6e1.slice/crio-aa51ea49dfaa82ad428ca1a3250b18997d1f29013016930df277f386ec07cfcf WatchSource:0}: Error finding container aa51ea49dfaa82ad428ca1a3250b18997d1f29013016930df277f386ec07cfcf: Status 404 returned error can't find the container with id aa51ea49dfaa82ad428ca1a3250b18997d1f29013016930df277f386ec07cfcf Jul 07 19:27:48.589695 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:48.589658 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" event={"ID":"da67d31c-c3df-4723-b097-039e1a52c6e1","Type":"ContainerStarted","Data":"aa51ea49dfaa82ad428ca1a3250b18997d1f29013016930df277f386ec07cfcf"} Jul 07 19:27:49.976531 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:49.976497 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:50.028574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.028535 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-56f5l"] Jul 07 19:27:50.031247 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.031232 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:27:50.034683 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.034633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zc47v\"" Jul 07 19:27:50.052326 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.052297 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-56f5l"] Jul 07 19:27:50.120959 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.112298 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxvtk\" (UniqueName: \"kubernetes.io/projected/166367b2-93a7-4900-b08a-c7707545bdf2-kube-api-access-pxvtk\") pod \"authorino-operator-75cc8d5cb7-56f5l\" (UID: \"166367b2-93a7-4900-b08a-c7707545bdf2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:27:50.222363 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.222319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxvtk\" (UniqueName: \"kubernetes.io/projected/166367b2-93a7-4900-b08a-c7707545bdf2-kube-api-access-pxvtk\") pod \"authorino-operator-75cc8d5cb7-56f5l\" (UID: \"166367b2-93a7-4900-b08a-c7707545bdf2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:27:50.237817 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.237759 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxvtk\" (UniqueName: \"kubernetes.io/projected/166367b2-93a7-4900-b08a-c7707545bdf2-kube-api-access-pxvtk\") pod \"authorino-operator-75cc8d5cb7-56f5l\" (UID: \"166367b2-93a7-4900-b08a-c7707545bdf2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:27:50.341389 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.341362 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:27:50.486930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.486890 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-56f5l"] Jul 07 19:27:50.487104 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:27:50.487032 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166367b2_93a7_4900_b08a_c7707545bdf2.slice/crio-8482ec07975d5f27058ac091a4adc82d18ab189b5c23e2a29746e63a032752a6 WatchSource:0}: Error finding container 8482ec07975d5f27058ac091a4adc82d18ab189b5c23e2a29746e63a032752a6: Status 404 returned error can't find the container with id 8482ec07975d5f27058ac091a4adc82d18ab189b5c23e2a29746e63a032752a6 Jul 07 19:27:50.607391 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.607330 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" event={"ID":"166367b2-93a7-4900-b08a-c7707545bdf2","Type":"ContainerStarted","Data":"8482ec07975d5f27058ac091a4adc82d18ab189b5c23e2a29746e63a032752a6"} Jul 07 19:27:50.929268 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:50.929191 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:53.184685 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:53.184646 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:54.570958 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.570930 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2tm9k" Jul 07 19:27:54.630989 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.630961 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" event={"ID":"da67d31c-c3df-4723-b097-039e1a52c6e1","Type":"ContainerStarted","Data":"cfc7c64dfe14775cad9b64d7c74c4f3e212094d385ccad515b9f8290cc9ae3ce"} Jul 07 19:27:54.631130 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.631054 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:27:54.632659 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.632636 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" event={"ID":"166367b2-93a7-4900-b08a-c7707545bdf2","Type":"ContainerStarted","Data":"b11f3b8d0a3c4fbc0c88aadb3269ab95c03eda2db97865d58e075a008dae72f4"} Jul 07 19:27:54.632788 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.632772 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:27:54.649994 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.649954 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" podStartSLOduration=2.72419239 podStartE2EDuration="8.64993969s" totalImagesPullingTime="5.9257473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:27:54.649226686 +0000 UTC m=+420.551105222" watchObservedRunningTime="2026-07-07 19:27:54.64993969 +0000 UTC m=+420.551818269" Jul 07 19:27:54.667821 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.667780 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" podStartSLOduration=2.522189821 podStartE2EDuration="5.667768918s" totalImagesPullingTime="3.145579097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:27:54.664808918 +0000 UTC m=+420.566687472" watchObservedRunningTime="2026-07-07 19:27:54.667768918 +0000 UTC m=+420.569647454" Jul 07 19:27:54.736863 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.736830 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:54.839069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:54.838991 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:56.292276 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:56.292240 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:56.386313 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:56.386280 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:58.335500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:58.335466 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:58.430286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:58.430242 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:58.515548 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:58.515510 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:27:58.616736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:27:58.616633 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:01.559193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:01.559161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:05.639000 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:05.638969 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-56f5l" Jul 07 19:28:05.639500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:05.639048 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.340101 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.340068 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j"] Jul 07 19:28:07.343833 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.343815 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.361254 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.361229 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j"] Jul 07 19:28:07.462420 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.462374 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rg62\" (UniqueName: \"kubernetes.io/projected/9d420102-54c7-49cb-9c26-bf7df4bac60b-kube-api-access-6rg62\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" (UID: \"9d420102-54c7-49cb-9c26-bf7df4bac60b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.462585 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.462486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d420102-54c7-49cb-9c26-bf7df4bac60b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" (UID: \"9d420102-54c7-49cb-9c26-bf7df4bac60b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.477919 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.477882 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j"] Jul 07 19:28:07.478173 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.478148 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-6rg62], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" podUID="9d420102-54c7-49cb-9c26-bf7df4bac60b" Jul 07 19:28:07.496426 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.496397 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j"] Jul 07 19:28:07.513277 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.513253 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j"] Jul 07 19:28:07.526681 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.526649 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w"] Jul 07 19:28:07.527065 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.527035 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" containerName="manager" containerID="cri-o://cfc7c64dfe14775cad9b64d7c74c4f3e212094d385ccad515b9f8290cc9ae3ce" gracePeriod=2 Jul 07 19:28:07.528318 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.528297 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg"] Jul 07 19:28:07.531116 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.531094 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.543746 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.543726 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w"] Jul 07 19:28:07.546431 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.546411 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg"] Jul 07 19:28:07.546633 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.546606 2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"da67d31c-c3df-4723-b097-039e1a52c6e1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\": pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.550109 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.549935 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.559159 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.559136 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt"] Jul 07 19:28:07.559508 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.559494 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" containerName="manager" Jul 07 19:28:07.559508 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.559508 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" containerName="manager" Jul 07 19:28:07.559629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.559576 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" containerName="manager" Jul 07 19:28:07.561533 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.561511 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.563347 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.563321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d420102-54c7-49cb-9c26-bf7df4bac60b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" (UID: \"9d420102-54c7-49cb-9c26-bf7df4bac60b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.563514 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.563479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rg62\" (UniqueName: \"kubernetes.io/projected/9d420102-54c7-49cb-9c26-bf7df4bac60b-kube-api-access-6rg62\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" (UID: \"9d420102-54c7-49cb-9c26-bf7df4bac60b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.563742 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.563723 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d420102-54c7-49cb-9c26-bf7df4bac60b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" (UID: \"9d420102-54c7-49cb-9c26-bf7df4bac60b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.569950 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.569934 2561 projected.go:196] Error preparing data for projected volume kube-api-access-6rg62 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-b8t9j" not found Jul 07 19:28:07.570039 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.570007 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d420102-54c7-49cb-9c26-bf7df4bac60b-kube-api-access-6rg62 podName:9d420102-54c7-49cb-9c26-bf7df4bac60b nodeName:}" failed. No retries permitted until 2026-07-07 19:28:08.069994126 +0000 UTC m=+433.971872642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6rg62" (UniqueName: "kubernetes.io/projected/9d420102-54c7-49cb-9c26-bf7df4bac60b-kube-api-access-6rg62") pod "kuadrant-operator-controller-manager-55467d9f7-b8t9j" (UID: "9d420102-54c7-49cb-9c26-bf7df4bac60b") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-b8t9j" not found Jul 07 19:28:07.585567 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.585491 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.588753 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.588728 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt"] Jul 07 19:28:07.622848 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.622820 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:07.664092 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.664065 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk78c\" (UniqueName: \"kubernetes.io/projected/6fe90dec-3f26-4cdd-96f0-679605b480d6-kube-api-access-jk78c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsgmt\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.664201 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.664171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc1fc652-4707-435e-a09d-dea696a8fae8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sf7bg\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.664247 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.664229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjb2n\" (UniqueName: \"kubernetes.io/projected/cc1fc652-4707-435e-a09d-dea696a8fae8-kube-api-access-fjb2n\") pod \"kuadrant-operator-controller-manager-86878c769f-sf7bg\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.664300 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.664274 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fe90dec-3f26-4cdd-96f0-679605b480d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsgmt\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.702770 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.702694 2561 generic.go:356] "Generic (PLEG): container finished" podID="da67d31c-c3df-4723-b097-039e1a52c6e1" containerID="cfc7c64dfe14775cad9b64d7c74c4f3e212094d385ccad515b9f8290cc9ae3ce" exitCode=0 Jul 07 19:28:07.702957 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.702940 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.705415 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.705228 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="9d420102-54c7-49cb-9c26-bf7df4bac60b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.707322 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.707291 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.709974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.709957 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.712061 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.712034 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.713854 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.713830 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="9d420102-54c7-49cb-9c26-bf7df4bac60b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.725205 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.725164 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:07.764808 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.764781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fe90dec-3f26-4cdd-96f0-679605b480d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsgmt\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.764927 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.764904 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk78c\" (UniqueName: \"kubernetes.io/projected/6fe90dec-3f26-4cdd-96f0-679605b480d6-kube-api-access-jk78c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsgmt\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.764996 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.764961 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc1fc652-4707-435e-a09d-dea696a8fae8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sf7bg\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.765056 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.765000 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjb2n\" (UniqueName: \"kubernetes.io/projected/cc1fc652-4707-435e-a09d-dea696a8fae8-kube-api-access-fjb2n\") pod \"kuadrant-operator-controller-manager-86878c769f-sf7bg\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.765203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.765175 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fe90dec-3f26-4cdd-96f0-679605b480d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsgmt\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.765368 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.765344 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc1fc652-4707-435e-a09d-dea696a8fae8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sf7bg\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.770337 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.770320 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.772308 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.772286 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fzz8w\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:07.776306 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:07.776281 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="9d420102-54c7-49cb-9c26-bf7df4bac60b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:07.778027 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.778007 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk78c\" (UniqueName: \"kubernetes.io/projected/6fe90dec-3f26-4cdd-96f0-679605b480d6-kube-api-access-jk78c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsgmt\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.779862 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.779845 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjb2n\" (UniqueName: \"kubernetes.io/projected/cc1fc652-4707-435e-a09d-dea696a8fae8-kube-api-access-fjb2n\") pod \"kuadrant-operator-controller-manager-86878c769f-sf7bg\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.843270 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.843237 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:07.865970 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.865946 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d420102-54c7-49cb-9c26-bf7df4bac60b-extensions-socket-volume\") pod \"9d420102-54c7-49cb-9c26-bf7df4bac60b\" (UID: \"9d420102-54c7-49cb-9c26-bf7df4bac60b\") " Jul 07 19:28:07.866107 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.866007 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da67d31c-c3df-4723-b097-039e1a52c6e1-extensions-socket-volume\") pod \"da67d31c-c3df-4723-b097-039e1a52c6e1\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " Jul 07 19:28:07.866107 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.866035 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc72m\" (UniqueName: \"kubernetes.io/projected/da67d31c-c3df-4723-b097-039e1a52c6e1-kube-api-access-wc72m\") pod \"da67d31c-c3df-4723-b097-039e1a52c6e1\" (UID: \"da67d31c-c3df-4723-b097-039e1a52c6e1\") " Jul 07 19:28:07.866242 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.866218 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d420102-54c7-49cb-9c26-bf7df4bac60b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9d420102-54c7-49cb-9c26-bf7df4bac60b" (UID: "9d420102-54c7-49cb-9c26-bf7df4bac60b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:28:07.866362 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.866348 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rg62\" (UniqueName: \"kubernetes.io/projected/9d420102-54c7-49cb-9c26-bf7df4bac60b-kube-api-access-6rg62\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:07.866448 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.866363 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d420102-54c7-49cb-9c26-bf7df4bac60b-extensions-socket-volume\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:07.866506 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.866449 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da67d31c-c3df-4723-b097-039e1a52c6e1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "da67d31c-c3df-4723-b097-039e1a52c6e1" (UID: "da67d31c-c3df-4723-b097-039e1a52c6e1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:28:07.868130 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.868093 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da67d31c-c3df-4723-b097-039e1a52c6e1-kube-api-access-wc72m" (OuterVolumeSpecName: "kube-api-access-wc72m") pod "da67d31c-c3df-4723-b097-039e1a52c6e1" (UID: "da67d31c-c3df-4723-b097-039e1a52c6e1"). InnerVolumeSpecName "kube-api-access-wc72m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:28:07.906688 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.906651 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:07.967768 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.967734 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da67d31c-c3df-4723-b097-039e1a52c6e1-extensions-socket-volume\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:07.967768 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.967768 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc72m\" (UniqueName: \"kubernetes.io/projected/da67d31c-c3df-4723-b097-039e1a52c6e1-kube-api-access-wc72m\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:07.986024 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:07.985852 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg"] Jul 07 19:28:07.988934 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:28:07.988899 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1fc652_4707_435e_a09d_dea696a8fae8.slice/crio-325ac20f437a68a150a5a36119cf5389f08c6b7c7f993206538fa099b92b60a5 WatchSource:0}: Error finding container 325ac20f437a68a150a5a36119cf5389f08c6b7c7f993206538fa099b92b60a5: Status 404 returned error can't find the container with id 325ac20f437a68a150a5a36119cf5389f08c6b7c7f993206538fa099b92b60a5 Jul 07 19:28:08.069729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.069704 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt"] Jul 07 19:28:08.071466 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:28:08.071441 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe90dec_3f26_4cdd_96f0_679605b480d6.slice/crio-b865bb3b722d7e22718b219d78284f23b32ddbed50b3455958c6dc6b22c293d7 WatchSource:0}: Error finding container b865bb3b722d7e22718b219d78284f23b32ddbed50b3455958c6dc6b22c293d7: Status 404 returned error can't find the container with id b865bb3b722d7e22718b219d78284f23b32ddbed50b3455958c6dc6b22c293d7 Jul 07 19:28:08.623058 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.623027 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d420102-54c7-49cb-9c26-bf7df4bac60b" path="/var/lib/kubelet/pods/9d420102-54c7-49cb-9c26-bf7df4bac60b/volumes" Jul 07 19:28:08.623434 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.623262 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da67d31c-c3df-4723-b097-039e1a52c6e1" path="/var/lib/kubelet/pods/da67d31c-c3df-4723-b097-039e1a52c6e1/volumes" Jul 07 19:28:08.712252 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.712210 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" event={"ID":"6fe90dec-3f26-4cdd-96f0-679605b480d6","Type":"ContainerStarted","Data":"5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644"} Jul 07 19:28:08.712473 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.712275 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" event={"ID":"6fe90dec-3f26-4cdd-96f0-679605b480d6","Type":"ContainerStarted","Data":"b865bb3b722d7e22718b219d78284f23b32ddbed50b3455958c6dc6b22c293d7"} Jul 07 19:28:08.712544 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.712499 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:08.715017 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.714981 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" event={"ID":"cc1fc652-4707-435e-a09d-dea696a8fae8","Type":"ContainerStarted","Data":"e9e8f3d709487cad281de95fe8a0c9ca85b4123aa55759bdc38a397f05020cba"} Jul 07 19:28:08.715147 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.715030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" event={"ID":"cc1fc652-4707-435e-a09d-dea696a8fae8","Type":"ContainerStarted","Data":"325ac20f437a68a150a5a36119cf5389f08c6b7c7f993206538fa099b92b60a5"} Jul 07 19:28:08.715147 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.715080 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:08.716793 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.716772 2561 scope.go:117] "RemoveContainer" containerID="cfc7c64dfe14775cad9b64d7c74c4f3e212094d385ccad515b9f8290cc9ae3ce" Jul 07 19:28:08.716937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.716774 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:08.717017 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.716779 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fzz8w" Jul 07 19:28:08.736898 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.736850 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" podStartSLOduration=1.7368346620000001 podStartE2EDuration="1.736834662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:28:08.734487675 +0000 UTC m=+434.636366214" watchObservedRunningTime="2026-07-07 19:28:08.736834662 +0000 UTC m=+434.638713196" Jul 07 19:28:08.737032 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:08.736926 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b8t9j\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="9d420102-54c7-49cb-9c26-bf7df4bac60b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b8t9j" Jul 07 19:28:08.763596 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:08.763551 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" podStartSLOduration=1.763533091 podStartE2EDuration="1.763533091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:28:08.762453265 +0000 UTC m=+434.664331814" watchObservedRunningTime="2026-07-07 19:28:08.763533091 +0000 UTC m=+434.665411625" Jul 07 19:28:10.547292 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:10.547256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:10.648702 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:10.648667 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:13.145101 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:13.145066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:13.249845 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:13.249806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:16.950814 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:16.950729 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:17.055161 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:17.055122 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:19.724372 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:19.724340 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:28:19.724838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:19.724427 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:19.828678 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:19.828644 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt"] Jul 07 19:28:19.828887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:19.828863 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" podUID="6fe90dec-3f26-4cdd-96f0-679605b480d6" containerName="manager" containerID="cri-o://5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644" gracePeriod=10 Jul 07 19:28:20.081934 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.081911 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:20.166719 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.166690 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk78c\" (UniqueName: \"kubernetes.io/projected/6fe90dec-3f26-4cdd-96f0-679605b480d6-kube-api-access-jk78c\") pod \"6fe90dec-3f26-4cdd-96f0-679605b480d6\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " Jul 07 19:28:20.168726 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.168693 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe90dec-3f26-4cdd-96f0-679605b480d6-kube-api-access-jk78c" (OuterVolumeSpecName: "kube-api-access-jk78c") pod "6fe90dec-3f26-4cdd-96f0-679605b480d6" (UID: "6fe90dec-3f26-4cdd-96f0-679605b480d6"). InnerVolumeSpecName "kube-api-access-jk78c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:28:20.267261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.267177 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fe90dec-3f26-4cdd-96f0-679605b480d6-extensions-socket-volume\") pod \"6fe90dec-3f26-4cdd-96f0-679605b480d6\" (UID: \"6fe90dec-3f26-4cdd-96f0-679605b480d6\") " Jul 07 19:28:20.267444 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.267317 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk78c\" (UniqueName: \"kubernetes.io/projected/6fe90dec-3f26-4cdd-96f0-679605b480d6-kube-api-access-jk78c\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:20.267559 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.267535 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe90dec-3f26-4cdd-96f0-679605b480d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6fe90dec-3f26-4cdd-96f0-679605b480d6" (UID: "6fe90dec-3f26-4cdd-96f0-679605b480d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:28:20.367745 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.367704 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fe90dec-3f26-4cdd-96f0-679605b480d6-extensions-socket-volume\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:28:20.775613 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.775573 2561 generic.go:356] "Generic (PLEG): container finished" podID="6fe90dec-3f26-4cdd-96f0-679605b480d6" containerID="5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644" exitCode=0 Jul 07 19:28:20.776069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.775622 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" event={"ID":"6fe90dec-3f26-4cdd-96f0-679605b480d6","Type":"ContainerDied","Data":"5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644"} Jul 07 19:28:20.776069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.775717 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" event={"ID":"6fe90dec-3f26-4cdd-96f0-679605b480d6","Type":"ContainerDied","Data":"b865bb3b722d7e22718b219d78284f23b32ddbed50b3455958c6dc6b22c293d7"} Jul 07 19:28:20.776069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.775755 2561 scope.go:117] "RemoveContainer" containerID="5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644" Jul 07 19:28:20.776069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.775945 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt" Jul 07 19:28:20.788945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.788927 2561 scope.go:117] "RemoveContainer" containerID="5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644" Jul 07 19:28:20.789230 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:28:20.789201 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644\": container with ID starting with 5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644 not found: ID does not exist" containerID="5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644" Jul 07 19:28:20.789289 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.789244 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644"} err="failed to get container status \"5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644\": rpc error: code = NotFound desc = could not find container \"5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644\": container with ID starting with 5377e702c37ab2bef0c97b56a5b77ab9d6007900d3c2235c93ddbf62babe8644 not found: ID does not exist" Jul 07 19:28:20.796564 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.796542 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt"] Jul 07 19:28:20.804672 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:20.804650 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsgmt"] Jul 07 19:28:22.623707 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:22.623675 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fe90dec-3f26-4cdd-96f0-679605b480d6" path="/var/lib/kubelet/pods/6fe90dec-3f26-4cdd-96f0-679605b480d6/volumes" Jul 07 19:28:26.005357 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:26.005312 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:26.172767 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:26.172734 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:28.966516 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:28.966483 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:29.066804 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:29.066764 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:36.214029 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.213992 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64"] Jul 07 19:28:36.214416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.214360 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe90dec-3f26-4cdd-96f0-679605b480d6" containerName="manager" Jul 07 19:28:36.214416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.214371 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe90dec-3f26-4cdd-96f0-679605b480d6" containerName="manager" Jul 07 19:28:36.214494 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.214452 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fe90dec-3f26-4cdd-96f0-679605b480d6" containerName="manager" Jul 07 19:28:36.221001 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.220977 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.223431 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.223409 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8ppsp\"" Jul 07 19:28:36.229738 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.229673 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64"] Jul 07 19:28:36.235076 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.235053 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:36.304150 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304111 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304343 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304163 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304343 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304230 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304343 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304343 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304302 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp4h8\" (UniqueName: \"kubernetes.io/projected/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-kube-api-access-vp4h8\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304413 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304458 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.304574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.304531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405313 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405527 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405327 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp4h8\" (UniqueName: \"kubernetes.io/projected/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-kube-api-access-vp4h8\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405527 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405361 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405527 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405427 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405527 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405535 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405565 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.405978 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.405952 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.406063 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.406044 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.406119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.406102 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.406339 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.406313 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.406435 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.406357 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.408158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.408129 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.408423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.408400 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.417311 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.417290 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.417600 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.417584 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp4h8\" (UniqueName: \"kubernetes.io/projected/32aa69e5-8cda-4b8f-9ffc-e297e8e80e48-kube-api-access-vp4h8\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zcs64\" (UID: \"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.516648 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.516549 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:36.533401 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.533355 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:36.679227 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.679203 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64"] Jul 07 19:28:36.680847 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:28:36.680820 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32aa69e5_8cda_4b8f_9ffc_e297e8e80e48.slice/crio-a6a01e13b4215f238615fb8785a88d32b8a3c5eedc985efcb624f069ed87ac39 WatchSource:0}: Error finding container a6a01e13b4215f238615fb8785a88d32b8a3c5eedc985efcb624f069ed87ac39: Status 404 returned error can't find the container with id a6a01e13b4215f238615fb8785a88d32b8a3c5eedc985efcb624f069ed87ac39 Jul 07 19:28:36.682940 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.682906 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:28:36.683033 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.682972 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:28:36.683033 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.683000 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:28:36.852926 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.852834 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" event={"ID":"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48","Type":"ContainerStarted","Data":"15d41d0d4b7c6e9dd0fe8a39df800b9972dd67722afec092712d8985aaeab05c"} Jul 07 19:28:36.852926 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.852891 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" event={"ID":"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48","Type":"ContainerStarted","Data":"a6a01e13b4215f238615fb8785a88d32b8a3c5eedc985efcb624f069ed87ac39"} Jul 07 19:28:36.875472 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:36.875419 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" podStartSLOduration=0.875402422 podStartE2EDuration="875.402422ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:28:36.874173588 +0000 UTC m=+462.776052124" watchObservedRunningTime="2026-07-07 19:28:36.875402422 +0000 UTC m=+462.777280956" Jul 07 19:28:37.534214 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:37.534185 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:37.538704 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:37.538681 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:37.858579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:37.858500 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:37.859630 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:37.859611 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:28:38.037441 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.037375 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:38.180030 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.179948 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:38.350412 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.350358 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:38.712063 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.712030 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:28:38.714906 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.714880 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.718141 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.718114 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f5gcm\"" Jul 07 19:28:38.718263 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.718157 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 19:28:38.726466 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.726439 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:28:38.742802 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.742769 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:28:38.828516 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.828479 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p2r2\" (UniqueName: \"kubernetes.io/projected/910de4bf-d537-4590-8bc6-18da98b5e325-kube-api-access-2p2r2\") pod \"limitador-limitador-54d6d988d5-pghc4\" (UID: \"910de4bf-d537-4590-8bc6-18da98b5e325\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.828681 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.828551 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/910de4bf-d537-4590-8bc6-18da98b5e325-config-file\") pod \"limitador-limitador-54d6d988d5-pghc4\" (UID: \"910de4bf-d537-4590-8bc6-18da98b5e325\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.929125 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.929089 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/910de4bf-d537-4590-8bc6-18da98b5e325-config-file\") pod \"limitador-limitador-54d6d988d5-pghc4\" (UID: \"910de4bf-d537-4590-8bc6-18da98b5e325\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.929300 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.929208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p2r2\" (UniqueName: \"kubernetes.io/projected/910de4bf-d537-4590-8bc6-18da98b5e325-kube-api-access-2p2r2\") pod \"limitador-limitador-54d6d988d5-pghc4\" (UID: \"910de4bf-d537-4590-8bc6-18da98b5e325\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.929813 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.929790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/910de4bf-d537-4590-8bc6-18da98b5e325-config-file\") pod \"limitador-limitador-54d6d988d5-pghc4\" (UID: \"910de4bf-d537-4590-8bc6-18da98b5e325\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.949687 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.949660 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p2r2\" (UniqueName: \"kubernetes.io/projected/910de4bf-d537-4590-8bc6-18da98b5e325-kube-api-access-2p2r2\") pod \"limitador-limitador-54d6d988d5-pghc4\" (UID: \"910de4bf-d537-4590-8bc6-18da98b5e325\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:38.980690 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:38.980615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:39.026869 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:39.026838 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:39.080350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:39.080254 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:39.161892 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:39.161870 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:28:39.163980 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:28:39.163952 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910de4bf_d537_4590_8bc6_18da98b5e325.slice/crio-cc38a7d8e0555f092d17d709113fe9ee58c150570afa5430537e67529fdd9e65 WatchSource:0}: Error finding container cc38a7d8e0555f092d17d709113fe9ee58c150570afa5430537e67529fdd9e65: Status 404 returned error can't find the container with id cc38a7d8e0555f092d17d709113fe9ee58c150570afa5430537e67529fdd9e65 Jul 07 19:28:39.871221 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:39.871184 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" event={"ID":"910de4bf-d537-4590-8bc6-18da98b5e325","Type":"ContainerStarted","Data":"cc38a7d8e0555f092d17d709113fe9ee58c150570afa5430537e67529fdd9e65"} Jul 07 19:28:41.883765 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:41.883684 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" event={"ID":"910de4bf-d537-4590-8bc6-18da98b5e325","Type":"ContainerStarted","Data":"1c82c5ed6e3a490a540cf9a42807700bd820e114f759a99d5195381daeaa2084"} Jul 07 19:28:41.884113 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:41.883799 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:52.889444 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:52.889410 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" Jul 07 19:28:52.910389 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:52.910329 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pghc4" podStartSLOduration=12.498803406 podStartE2EDuration="14.910313784s" totalImagesPullingTime="2.411510378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:28:41.909580183 +0000 UTC m=+467.811458717" watchObservedRunningTime="2026-07-07 19:28:52.910313784 +0000 UTC m=+478.812192318" Jul 07 19:28:54.484788 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:54.484752 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:28:59.716543 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:28:59.716509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:07.458015 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:07.457976 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:07.709400 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:07.709298 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:14.334346 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:14.334301 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:14.438804 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:14.438768 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:14.541976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:14.541939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:14.640923 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:14.640830 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:17.851800 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:17.851764 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:17.954759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:17.954730 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:50.381888 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:50.381806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:50.854180 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:50.854145 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:51.063436 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:51.063409 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:51.384806 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:51.384769 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:51.481327 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:51.481291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:29:51.670116 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:51.670038 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:29:59.260980 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:59.260907 2561 generic.go:356] "Generic (PLEG): container finished" podID="32aa69e5-8cda-4b8f-9ffc-e297e8e80e48" containerID="15d41d0d4b7c6e9dd0fe8a39df800b9972dd67722afec092712d8985aaeab05c" exitCode=0 Jul 07 19:29:59.260980 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:59.260949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" event={"ID":"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48","Type":"ContainerDied","Data":"15d41d0d4b7c6e9dd0fe8a39df800b9972dd67722afec092712d8985aaeab05c"} Jul 07 19:29:59.261460 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:59.261367 2561 scope.go:117] "RemoveContainer" containerID="15d41d0d4b7c6e9dd0fe8a39df800b9972dd67722afec092712d8985aaeab05c" Jul 07 19:29:59.262406 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:59.262354 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:29:59.262504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:59.262446 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:29:59.262504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:29:59.262477 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:30:00.273689 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:00.273653 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" event={"ID":"32aa69e5-8cda-4b8f-9ffc-e297e8e80e48","Type":"ContainerStarted","Data":"a922cc71503d340598566c58f51d6ced8110b637359455334bb14bdc8c9b0130"} Jul 07 19:30:00.534798 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:00.534651 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:30:00.535078 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:00.535054 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zcs64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 19:30:00.535244 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:00.535221 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" podUID="32aa69e5-8cda-4b8f-9ffc-e297e8e80e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 19:30:01.385965 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:01.385933 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:01.485822 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:01.485785 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:01.533821 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:01.533780 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zcs64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 19:30:01.533985 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:01.533846 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" podUID="32aa69e5-8cda-4b8f-9ffc-e297e8e80e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 19:30:02.534188 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:02.534152 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zcs64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 19:30:02.534779 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:02.534749 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" podUID="32aa69e5-8cda-4b8f-9ffc-e297e8e80e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 19:30:03.534518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:03.534485 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zcs64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 19:30:03.535113 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:03.535062 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" podUID="32aa69e5-8cda-4b8f-9ffc-e297e8e80e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 19:30:04.538221 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:04.538189 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:30:04.538620 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:04.538552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:30:04.539244 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:04.539226 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zcs64" Jul 07 19:30:11.241828 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:11.241774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:11.719755 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:11.719722 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:30:17.126494 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:17.126460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:17.786841 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:17.786806 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:30:40.505959 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:40.505927 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:41.479692 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:41.479661 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:30:41.656575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:41.656542 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:41.757871 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:41.757777 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:30:54.528177 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:54.528139 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:30:54.530263 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:30:54.530240 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:31:01.129570 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:01.129529 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:31:02.582881 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:02.582845 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:31:06.405580 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:06.405542 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:31:09.179016 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:09.178983 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:31:12.390395 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:12.390341 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:31:17.808775 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:17.808690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:31:20.284818 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:20.284786 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:31:50.490685 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:50.490642 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:31:50.597109 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:31:50.597076 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:12.716334 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:12.716279 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:12.985597 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:12.985512 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:13.902625 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:13.902594 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:14.022015 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:14.021981 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:20.338831 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:20.338786 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:41.425329 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:41.425293 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:32:50.361881 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:50.361798 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:50.449529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:50.449492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:32:52.685198 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:52.685165 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:32:56.947581 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:32:56.947539 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:33:01.573156 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:01.573123 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:33:09.428780 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:09.428742 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:33:18.772398 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:18.772348 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:33:24.888970 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:24.888942 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:33:26.199841 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:26.199805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:33:26.277082 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:26.277056 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:33:47.891762 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:47.891728 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:33:54.312817 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:54.312777 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:33:59.185133 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:33:59.185097 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:34:07.047074 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:07.047043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:07.144646 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:07.144608 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:07.303041 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:07.302957 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:13.365122 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:13.365087 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:13.478897 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:13.478858 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:13.568022 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:13.567979 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:13.783108 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:13.783078 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:19.986203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:19.986123 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:20.084066 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:20.084028 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:34:36.280321 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:36.280281 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:34:43.681615 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:43.681581 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:34:53.685312 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:53.685279 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:34:59.985357 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:34:59.985319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:35:08.685926 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:08.685889 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:35:17.572276 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:17.572241 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:35:29.281275 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:29.281235 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:35:38.785979 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:38.785948 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:35:51.029580 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:51.029485 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:35:54.561267 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:54.561232 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:35:54.563147 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:35:54.563125 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:36:01.771462 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:36:01.771426 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:36:10.283846 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:36:10.283812 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:36:33.577997 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:36:33.577959 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:36:44.400211 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:36:44.400178 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:37:02.770756 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:37:02.770722 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:37:11.487732 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:37:11.487692 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:37:20.972643 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:37:20.972568 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:37:29.880626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:37:29.880591 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:37:38.583904 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:37:38.583868 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:38:05.983708 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:05.983671 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg"] Jul 07 19:38:05.984213 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:05.983896 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" podUID="cc1fc652-4707-435e-a09d-dea696a8fae8" containerName="manager" containerID="cri-o://e9e8f3d709487cad281de95fe8a0c9ca85b4123aa55759bdc38a397f05020cba" gracePeriod=10 Jul 07 19:38:08.592264 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.592234 2561 generic.go:356] "Generic (PLEG): container finished" podID="cc1fc652-4707-435e-a09d-dea696a8fae8" containerID="e9e8f3d709487cad281de95fe8a0c9ca85b4123aa55759bdc38a397f05020cba" exitCode=0 Jul 07 19:38:08.592688 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.592294 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" event={"ID":"cc1fc652-4707-435e-a09d-dea696a8fae8","Type":"ContainerDied","Data":"e9e8f3d709487cad281de95fe8a0c9ca85b4123aa55759bdc38a397f05020cba"} Jul 07 19:38:08.736049 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.736028 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:38:08.852199 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.852106 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjb2n\" (UniqueName: \"kubernetes.io/projected/cc1fc652-4707-435e-a09d-dea696a8fae8-kube-api-access-fjb2n\") pod \"cc1fc652-4707-435e-a09d-dea696a8fae8\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " Jul 07 19:38:08.852354 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.852295 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc1fc652-4707-435e-a09d-dea696a8fae8-extensions-socket-volume\") pod \"cc1fc652-4707-435e-a09d-dea696a8fae8\" (UID: \"cc1fc652-4707-435e-a09d-dea696a8fae8\") " Jul 07 19:38:08.852691 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.852668 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc1fc652-4707-435e-a09d-dea696a8fae8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cc1fc652-4707-435e-a09d-dea696a8fae8" (UID: "cc1fc652-4707-435e-a09d-dea696a8fae8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:38:08.854094 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.854077 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc1fc652-4707-435e-a09d-dea696a8fae8-kube-api-access-fjb2n" (OuterVolumeSpecName: "kube-api-access-fjb2n") pod "cc1fc652-4707-435e-a09d-dea696a8fae8" (UID: "cc1fc652-4707-435e-a09d-dea696a8fae8"). InnerVolumeSpecName "kube-api-access-fjb2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:38:08.952887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.952856 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc1fc652-4707-435e-a09d-dea696a8fae8-extensions-socket-volume\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:38:08.952887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:08.952883 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjb2n\" (UniqueName: \"kubernetes.io/projected/cc1fc652-4707-435e-a09d-dea696a8fae8-kube-api-access-fjb2n\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:38:09.598651 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:09.598623 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" Jul 07 19:38:09.599164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:09.598650 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg" event={"ID":"cc1fc652-4707-435e-a09d-dea696a8fae8","Type":"ContainerDied","Data":"325ac20f437a68a150a5a36119cf5389f08c6b7c7f993206538fa099b92b60a5"} Jul 07 19:38:09.599164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:09.598723 2561 scope.go:117] "RemoveContainer" containerID="e9e8f3d709487cad281de95fe8a0c9ca85b4123aa55759bdc38a397f05020cba" Jul 07 19:38:09.621601 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:09.621577 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg"] Jul 07 19:38:09.625690 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:09.625669 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sf7bg"] Jul 07 19:38:10.622621 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:38:10.622591 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1fc652-4707-435e-a09d-dea696a8fae8" path="/var/lib/kubelet/pods/cc1fc652-4707-435e-a09d-dea696a8fae8/volumes" Jul 07 19:39:07.190941 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.190901 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb"] Jul 07 19:39:07.191329 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.191256 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc1fc652-4707-435e-a09d-dea696a8fae8" containerName="manager" Jul 07 19:39:07.191329 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.191267 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1fc652-4707-435e-a09d-dea696a8fae8" containerName="manager" Jul 07 19:39:07.191430 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.191337 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc1fc652-4707-435e-a09d-dea696a8fae8" containerName="manager" Jul 07 19:39:07.194762 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.194744 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.200406 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.200372 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g5xpr\"" Jul 07 19:39:07.206682 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.206657 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb"] Jul 07 19:39:07.265682 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.265646 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06dbf8c6-8c5b-4381-ac52-2892c679f577-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gl7cb\" (UID: \"06dbf8c6-8c5b-4381-ac52-2892c679f577\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.265682 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.265684 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwfsl\" (UniqueName: \"kubernetes.io/projected/06dbf8c6-8c5b-4381-ac52-2892c679f577-kube-api-access-pwfsl\") pod \"kuadrant-operator-controller-manager-86878c769f-gl7cb\" (UID: \"06dbf8c6-8c5b-4381-ac52-2892c679f577\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.366759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.366725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06dbf8c6-8c5b-4381-ac52-2892c679f577-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gl7cb\" (UID: \"06dbf8c6-8c5b-4381-ac52-2892c679f577\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.366759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.366763 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwfsl\" (UniqueName: \"kubernetes.io/projected/06dbf8c6-8c5b-4381-ac52-2892c679f577-kube-api-access-pwfsl\") pod \"kuadrant-operator-controller-manager-86878c769f-gl7cb\" (UID: \"06dbf8c6-8c5b-4381-ac52-2892c679f577\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.367186 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.367166 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06dbf8c6-8c5b-4381-ac52-2892c679f577-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gl7cb\" (UID: \"06dbf8c6-8c5b-4381-ac52-2892c679f577\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.376671 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.376640 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwfsl\" (UniqueName: \"kubernetes.io/projected/06dbf8c6-8c5b-4381-ac52-2892c679f577-kube-api-access-pwfsl\") pod \"kuadrant-operator-controller-manager-86878c769f-gl7cb\" (UID: \"06dbf8c6-8c5b-4381-ac52-2892c679f577\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.504794 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.504711 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.661782 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.661743 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb"] Jul 07 19:39:07.667035 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:39:07.667005 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06dbf8c6_8c5b_4381_ac52_2892c679f577.slice/crio-1617aba9a6d7a85b8e649bee89d01ad628698afff9c3db2e5d37f81c588316d9 WatchSource:0}: Error finding container 1617aba9a6d7a85b8e649bee89d01ad628698afff9c3db2e5d37f81c588316d9: Status 404 returned error can't find the container with id 1617aba9a6d7a85b8e649bee89d01ad628698afff9c3db2e5d37f81c588316d9 Jul 07 19:39:07.864575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.864482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" event={"ID":"06dbf8c6-8c5b-4381-ac52-2892c679f577","Type":"ContainerStarted","Data":"dacf27fc87f87a7ec127026e2897c2ecf9945fbcd791905f0f5953492bd756fe"} Jul 07 19:39:07.864575 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.864536 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" event={"ID":"06dbf8c6-8c5b-4381-ac52-2892c679f577","Type":"ContainerStarted","Data":"1617aba9a6d7a85b8e649bee89d01ad628698afff9c3db2e5d37f81c588316d9"} Jul 07 19:39:07.864754 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.864598 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:07.886273 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:07.886218 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" podStartSLOduration=0.886202248 podStartE2EDuration="886.202248ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:39:07.885797342 +0000 UTC m=+1093.787675901" watchObservedRunningTime="2026-07-07 19:39:07.886202248 +0000 UTC m=+1093.788080783" Jul 07 19:39:18.570849 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:18.570819 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:39:18.872286 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:18.872203 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gl7cb" Jul 07 19:39:26.331343 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:26.331311 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:39:26.399690 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:26.399662 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:39:27.685997 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:27.685960 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:39:29.867994 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:29.867958 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:39:38.002764 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:38.002727 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:39:46.985021 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:46.984983 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:39:59.179063 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:39:59.179022 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:40:11.300364 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:11.300323 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:40:20.287230 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:20.287149 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:40:30.320433 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:30.320358 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:40:41.185624 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:41.185590 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:40:52.295836 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:52.295804 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:40:54.588976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:54.588948 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:40:54.593111 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:40:54.593089 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:41:01.085359 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:41:01.085325 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:41:11.177821 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:41:11.177791 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:41:36.382313 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:41:36.382280 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:41:43.885013 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:41:43.884979 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:41:53.874097 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:41:53.874012 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:42:02.870099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:02.870061 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:42:12.683219 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:12.683189 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:42:22.886200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:22.886168 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:42:34.581795 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:34.581763 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:42:43.777049 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:43.777017 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:42:51.905451 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:51.905412 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:42:53.386009 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:42:53.385974 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:43:02.886227 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:43:02.886193 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:43:11.590281 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:43:11.590238 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:43:25.487792 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:43:25.487716 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:43:34.379199 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:43:34.379160 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:43:43.898764 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:43:43.898728 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:43:55.374119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:43:55.374083 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:44:07.178944 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:44:07.178909 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:44:16.109221 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:44:16.109185 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:44:27.788391 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:44:27.788347 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:44:39.578410 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:44:39.578356 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:44:51.127396 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:44:51.127294 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:44:53.445617 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:44:53.445580 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:00.150134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.150097 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724225-w94bv"] Jul 07 19:45:00.153144 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.153127 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:00.156226 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.156203 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:45:00.183694 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.183669 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724225-w94bv"] Jul 07 19:45:00.217492 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.217463 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp7t2\" (UniqueName: \"kubernetes.io/projected/cbd0e593-2111-4ab0-9454-6c543d548717-kube-api-access-cp7t2\") pod \"maas-api-key-cleanup-29724225-w94bv\" (UID: \"cbd0e593-2111-4ab0-9454-6c543d548717\") " pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:00.318847 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.318806 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp7t2\" (UniqueName: \"kubernetes.io/projected/cbd0e593-2111-4ab0-9454-6c543d548717-kube-api-access-cp7t2\") pod \"maas-api-key-cleanup-29724225-w94bv\" (UID: \"cbd0e593-2111-4ab0-9454-6c543d548717\") " pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:00.328058 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.328027 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp7t2\" (UniqueName: \"kubernetes.io/projected/cbd0e593-2111-4ab0-9454-6c543d548717-kube-api-access-cp7t2\") pod \"maas-api-key-cleanup-29724225-w94bv\" (UID: \"cbd0e593-2111-4ab0-9454-6c543d548717\") " pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:00.463119 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.463077 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:00.597081 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.597018 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724225-w94bv"] Jul 07 19:45:00.600068 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:00.600038 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd0e593_2111_4ab0_9454_6c543d548717.slice/crio-97921c17263c55171f5fda6fce87bdad4e87046067722d4aa64d5b3e275f3659 WatchSource:0}: Error finding container 97921c17263c55171f5fda6fce87bdad4e87046067722d4aa64d5b3e275f3659: Status 404 returned error can't find the container with id 97921c17263c55171f5fda6fce87bdad4e87046067722d4aa64d5b3e275f3659 Jul 07 19:45:00.601925 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.601911 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:45:00.878137 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:00.878063 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:01.544149 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:01.544114 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" event={"ID":"cbd0e593-2111-4ab0-9454-6c543d548717","Type":"ContainerStarted","Data":"97921c17263c55171f5fda6fce87bdad4e87046067722d4aa64d5b3e275f3659"} Jul 07 19:45:04.274693 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:04.274652 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:45:06.574951 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:06.574915 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" event={"ID":"cbd0e593-2111-4ab0-9454-6c543d548717","Type":"ContainerStarted","Data":"a85b82c17958f3974deb2e49ec81b439a4ff5a87afa34e7434bbb14656ed5f8f"} Jul 07 19:45:06.592370 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:06.592323 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" podStartSLOduration=1.3762597300000001 podStartE2EDuration="6.592307002s" totalImagesPullingTime="5.216047272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:06.591441506 +0000 UTC m=+1452.493320066" watchObservedRunningTime="2026-07-07 19:45:06.592307002 +0000 UTC m=+1452.494185571" Jul 07 19:45:07.582747 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:07.582711 2561 generic.go:356] "Generic (PLEG): container finished" podID="cbd0e593-2111-4ab0-9454-6c543d548717" containerID="a85b82c17958f3974deb2e49ec81b439a4ff5a87afa34e7434bbb14656ed5f8f" exitCode=0 Jul 07 19:45:07.583147 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:07.582779 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" event={"ID":"cbd0e593-2111-4ab0-9454-6c543d548717","Type":"ContainerDied","Data":"a85b82c17958f3974deb2e49ec81b439a4ff5a87afa34e7434bbb14656ed5f8f"} Jul 07 19:45:08.717756 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:08.717730 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:08.799108 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:08.799077 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp7t2\" (UniqueName: \"kubernetes.io/projected/cbd0e593-2111-4ab0-9454-6c543d548717-kube-api-access-cp7t2\") pod \"cbd0e593-2111-4ab0-9454-6c543d548717\" (UID: \"cbd0e593-2111-4ab0-9454-6c543d548717\") " Jul 07 19:45:08.801214 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:08.801187 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd0e593-2111-4ab0-9454-6c543d548717-kube-api-access-cp7t2" (OuterVolumeSpecName: "kube-api-access-cp7t2") pod "cbd0e593-2111-4ab0-9454-6c543d548717" (UID: "cbd0e593-2111-4ab0-9454-6c543d548717"). InnerVolumeSpecName "kube-api-access-cp7t2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:08.900126 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:08.900043 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp7t2\" (UniqueName: \"kubernetes.io/projected/cbd0e593-2111-4ab0-9454-6c543d548717-kube-api-access-cp7t2\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:09.147403 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:09.147347 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:09.592874 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:09.592843 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" Jul 07 19:45:09.593073 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:09.592842 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724225-w94bv" event={"ID":"cbd0e593-2111-4ab0-9454-6c543d548717","Type":"ContainerDied","Data":"97921c17263c55171f5fda6fce87bdad4e87046067722d4aa64d5b3e275f3659"} Jul 07 19:45:09.593073 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:09.592987 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97921c17263c55171f5fda6fce87bdad4e87046067722d4aa64d5b3e275f3659" Jul 07 19:45:13.276753 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:13.276720 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:45:14.254127 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:14.254095 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:17.292681 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:17.292650 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:17.447250 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:17.447220 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:17.723569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:17.723540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:22.781021 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:22.780988 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:45:24.255166 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.255139 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:24.352355 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.352323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:24.455340 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.455313 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp"] Jul 07 19:45:24.455890 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.455870 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd0e593-2111-4ab0-9454-6c543d548717" containerName="cleanup" Jul 07 19:45:24.455983 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.455893 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd0e593-2111-4ab0-9454-6c543d548717" containerName="cleanup" Jul 07 19:45:24.456039 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.455985 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd0e593-2111-4ab0-9454-6c543d548717" containerName="cleanup" Jul 07 19:45:24.459011 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.458992 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.461417 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.461394 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-be0d7986-openshift-default-dockercfg-bc7xm\"" Jul 07 19:45:24.476616 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.476595 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp"] Jul 07 19:45:24.486057 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.486030 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:24.557863 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.557759 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:24.637490 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-envoy\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637490 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-istio-token\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637508 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg7kp\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-kube-api-access-hg7kp\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637528 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61f2c176-fd11-48ae-8737-6067672064d6-istiod-ca-cert\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637574 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-credential-socket\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637614 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61f2c176-fd11-48ae-8737-6067672064d6-istio-podinfo\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-certs\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-socket\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.637863 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.637753 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-data\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.738813 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.738779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-credential-socket\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.738974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.738841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61f2c176-fd11-48ae-8737-6067672064d6-istio-podinfo\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.738974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.738948 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-certs\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739058 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.738982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-socket\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739108 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739053 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-data\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739161 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739140 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-envoy\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739213 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739167 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-istio-token\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739213 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739186 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-credential-socket\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739207 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7kp\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-kube-api-access-hg7kp\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739245 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61f2c176-fd11-48ae-8737-6067672064d6-istiod-ca-cert\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739281 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-certs\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739522 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739398 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-socket\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.739599 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.739569 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-data\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.740095 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.740071 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61f2c176-fd11-48ae-8737-6067672064d6-istiod-ca-cert\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.741337 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.741317 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-envoy\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.741574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.741558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61f2c176-fd11-48ae-8737-6067672064d6-istio-podinfo\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.748045 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.748021 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-istio-token\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.748165 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.748145 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7kp\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-kube-api-access-hg7kp\") pod \"e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.769968 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.769947 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:24.911091 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.911066 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp"] Jul 07 19:45:24.913645 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:24.913620 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f2c176_fd11_48ae_8737_6067672064d6.slice/crio-56adf5145b89ed048be5121212ca047599b8d5f5f26e5c017d308a137aa75f2c WatchSource:0}: Error finding container 56adf5145b89ed048be5121212ca047599b8d5f5f26e5c017d308a137aa75f2c: Status 404 returned error can't find the container with id 56adf5145b89ed048be5121212ca047599b8d5f5f26e5c017d308a137aa75f2c Jul 07 19:45:24.915253 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.915214 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:24.915612 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.915310 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:24.915612 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:24.915353 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:25.670446 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.670409 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" event={"ID":"61f2c176-fd11-48ae-8737-6067672064d6","Type":"ContainerStarted","Data":"e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8"} Jul 07 19:45:25.670930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.670466 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" event={"ID":"61f2c176-fd11-48ae-8737-6067672064d6","Type":"ContainerStarted","Data":"56adf5145b89ed048be5121212ca047599b8d5f5f26e5c017d308a137aa75f2c"} Jul 07 19:45:25.673694 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.673668 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:25.703425 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.703369 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" podStartSLOduration=1.703355033 podStartE2EDuration="1.703355033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:25.69882178 +0000 UTC m=+1471.600700315" watchObservedRunningTime="2026-07-07 19:45:25.703355033 +0000 UTC m=+1471.605233568" Jul 07 19:45:25.768203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.768177 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd"] Jul 07 19:45:25.771532 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.771512 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:25.771660 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.771589 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:25.775205 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.775151 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:25.775915 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.775896 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-be0d7986-serving-cert\"" Jul 07 19:45:25.776177 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.776160 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:45:25.872351 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.872314 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd"] Jul 07 19:45:25.882713 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.882207 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:25.950878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.950844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:25.951031 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.950884 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkbnw\" (UniqueName: \"kubernetes.io/projected/093324ec-b860-4d33-b0a1-d06676a624cf-kube-api-access-kkbnw\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:25.951031 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.950945 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/093324ec-b860-4d33-b0a1-d06676a624cf-openshift-service-ca\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:25.968299 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:25.968261 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:26.052124 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.052098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.052260 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.052136 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkbnw\" (UniqueName: \"kubernetes.io/projected/093324ec-b860-4d33-b0a1-d06676a624cf-kube-api-access-kkbnw\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.052260 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.052190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/093324ec-b860-4d33-b0a1-d06676a624cf-openshift-service-ca\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.052260 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:26.052237 2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-be0d7986-serving-cert: secret "maas-api-e2e-ait-be0d7986-serving-cert" not found Jul 07 19:45:26.052367 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:26.052304 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls podName:093324ec-b860-4d33-b0a1-d06676a624cf nodeName:}" failed. No retries permitted until 2026-07-07 19:45:26.552286898 +0000 UTC m=+1472.454165431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls") pod "maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" (UID: "093324ec-b860-4d33-b0a1-d06676a624cf") : secret "maas-api-e2e-ait-be0d7986-serving-cert" not found Jul 07 19:45:26.052902 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.052884 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/093324ec-b860-4d33-b0a1-d06676a624cf-openshift-service-ca\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.061569 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.061543 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkbnw\" (UniqueName: \"kubernetes.io/projected/093324ec-b860-4d33-b0a1-d06676a624cf-kube-api-access-kkbnw\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.394804 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.394730 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:26.494267 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.494232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:26.558374 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.558343 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.560704 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.560682 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls\") pod \"maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.594925 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.594902 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:26.675321 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.675239 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:26.676212 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.676195 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:26.683467 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.683450 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:26.843612 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:26.843586 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd"] Jul 07 19:45:26.845789 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:26.845757 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093324ec_b860_4d33_b0a1_d06676a624cf.slice/crio-910cb3c09c5dca8a5518dde36f59ba81c52aa26990124e118dfec12a88ec0eae WatchSource:0}: Error finding container 910cb3c09c5dca8a5518dde36f59ba81c52aa26990124e118dfec12a88ec0eae: Status 404 returned error can't find the container with id 910cb3c09c5dca8a5518dde36f59ba81c52aa26990124e118dfec12a88ec0eae Jul 07 19:45:27.099647 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:27.099616 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:27.682476 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:27.682432 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" event={"ID":"093324ec-b860-4d33-b0a1-d06676a624cf","Type":"ContainerStarted","Data":"910cb3c09c5dca8a5518dde36f59ba81c52aa26990124e118dfec12a88ec0eae"} Jul 07 19:45:28.136451 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:28.136340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:29.693647 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:29.693613 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" event={"ID":"093324ec-b860-4d33-b0a1-d06676a624cf","Type":"ContainerStarted","Data":"8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973"} Jul 07 19:45:29.694038 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:29.693749 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:29.712994 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:29.712949 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" podStartSLOduration=2.499379122 podStartE2EDuration="4.712932497s" totalImagesPullingTime="2.213553375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:29.710941384 +0000 UTC m=+1475.612819932" watchObservedRunningTime="2026-07-07 19:45:29.712932497 +0000 UTC m=+1475.614811049" Jul 07 19:45:30.726303 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:30.726270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:30.890964 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:30.890931 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp"] Jul 07 19:45:30.891174 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:30.891144 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" podUID="61f2c176-fd11-48ae-8737-6067672064d6" containerName="istio-proxy" containerID="cri-o://e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8" gracePeriod=30 Jul 07 19:45:30.914390 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:30.914355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:31.056790 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:31.056708 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:31.160757 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:31.160721 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd"] Jul 07 19:45:31.195674 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:31.195649 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:31.703707 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:31.703670 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" podUID="093324ec-b860-4d33-b0a1-d06676a624cf" containerName="maas-api" containerID="cri-o://8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973" gracePeriod=30 Jul 07 19:45:31.952443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:31.952419 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:32.001456 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.001366 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkbnw\" (UniqueName: \"kubernetes.io/projected/093324ec-b860-4d33-b0a1-d06676a624cf-kube-api-access-kkbnw\") pod \"093324ec-b860-4d33-b0a1-d06676a624cf\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " Jul 07 19:45:32.001456 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.001419 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls\") pod \"093324ec-b860-4d33-b0a1-d06676a624cf\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " Jul 07 19:45:32.001639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.001466 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/093324ec-b860-4d33-b0a1-d06676a624cf-openshift-service-ca\") pod \"093324ec-b860-4d33-b0a1-d06676a624cf\" (UID: \"093324ec-b860-4d33-b0a1-d06676a624cf\") " Jul 07 19:45:32.001870 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.001843 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/093324ec-b860-4d33-b0a1-d06676a624cf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "093324ec-b860-4d33-b0a1-d06676a624cf" (UID: "093324ec-b860-4d33-b0a1-d06676a624cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:45:32.003500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.003463 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093324ec-b860-4d33-b0a1-d06676a624cf-kube-api-access-kkbnw" (OuterVolumeSpecName: "kube-api-access-kkbnw") pod "093324ec-b860-4d33-b0a1-d06676a624cf" (UID: "093324ec-b860-4d33-b0a1-d06676a624cf"). InnerVolumeSpecName "kube-api-access-kkbnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:32.003703 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.003681 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "093324ec-b860-4d33-b0a1-d06676a624cf" (UID: "093324ec-b860-4d33-b0a1-d06676a624cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:45:32.102252 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.102220 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/093324ec-b860-4d33-b0a1-d06676a624cf-maas-api-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:32.102252 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.102246 2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/093324ec-b860-4d33-b0a1-d06676a624cf-openshift-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:32.102252 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.102256 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkbnw\" (UniqueName: \"kubernetes.io/projected/093324ec-b860-4d33-b0a1-d06676a624cf-kube-api-access-kkbnw\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:32.709009 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.708972 2561 generic.go:356] "Generic (PLEG): container finished" podID="093324ec-b860-4d33-b0a1-d06676a624cf" containerID="8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973" exitCode=0 Jul 07 19:45:32.709197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.709024 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" Jul 07 19:45:32.709197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.709024 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" event={"ID":"093324ec-b860-4d33-b0a1-d06676a624cf","Type":"ContainerDied","Data":"8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973"} Jul 07 19:45:32.709197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.709090 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd" event={"ID":"093324ec-b860-4d33-b0a1-d06676a624cf","Type":"ContainerDied","Data":"910cb3c09c5dca8a5518dde36f59ba81c52aa26990124e118dfec12a88ec0eae"} Jul 07 19:45:32.709197 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.709122 2561 scope.go:117] "RemoveContainer" containerID="8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973" Jul 07 19:45:32.719262 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.719246 2561 scope.go:117] "RemoveContainer" containerID="8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973" Jul 07 19:45:32.719555 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:32.719537 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973\": container with ID starting with 8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973 not found: ID does not exist" containerID="8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973" Jul 07 19:45:32.719625 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.719565 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973"} err="failed to get container status \"8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973\": rpc error: code = NotFound desc = could not find container \"8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973\": container with ID starting with 8e465b67cd47c858199156ed468e860570d9fc8f54b1b08f110e46b27b3ba973 not found: ID does not exist" Jul 07 19:45:32.729072 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.729051 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd"] Jul 07 19:45:32.736064 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.736044 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-be0d7986-5b55bd95b-5whvd"] Jul 07 19:45:32.809427 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:32.809398 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:34.623701 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:34.623671 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="093324ec-b860-4d33-b0a1-d06676a624cf" path="/var/lib/kubelet/pods/093324ec-b860-4d33-b0a1-d06676a624cf/volumes" Jul 07 19:45:36.144976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.144955 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:36.235861 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.235836 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-istio-token\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236040 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.235887 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg7kp\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-kube-api-access-hg7kp\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236040 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.235914 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61f2c176-fd11-48ae-8737-6067672064d6-istiod-ca-cert\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236040 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236027 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61f2c176-fd11-48ae-8737-6067672064d6-istio-podinfo\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236211 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236066 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-certs\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236211 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236124 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-credential-socket\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236211 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236177 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-data\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236374 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236230 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-envoy\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236374 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236252 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-socket\") pod \"61f2c176-fd11-48ae-8737-6067672064d6\" (UID: \"61f2c176-fd11-48ae-8737-6067672064d6\") " Jul 07 19:45:36.236374 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236259 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61f2c176-fd11-48ae-8737-6067672064d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:45:36.236374 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236344 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:36.236631 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236403 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:36.236631 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236495 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:36.236631 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236614 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-credential-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.236631 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236617 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:36.236840 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236628 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-data\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.236840 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236669 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61f2c176-fd11-48ae-8737-6067672064d6-istiod-ca-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.236840 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.236684 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.238340 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.238320 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-kube-api-access-hg7kp" (OuterVolumeSpecName: "kube-api-access-hg7kp") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "kube-api-access-hg7kp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:36.238462 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.238342 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:36.238625 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.238598 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:36.238625 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.238611 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61f2c176-fd11-48ae-8737-6067672064d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61f2c176-fd11-48ae-8737-6067672064d6" (UID: "61f2c176-fd11-48ae-8737-6067672064d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 19:45:36.337187 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.337157 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61f2c176-fd11-48ae-8737-6067672064d6-istio-podinfo\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.337187 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.337182 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-istio-envoy\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.337187 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.337191 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61f2c176-fd11-48ae-8737-6067672064d6-workload-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.337434 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.337201 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-istio-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.337434 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.337210 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg7kp\" (UniqueName: \"kubernetes.io/projected/61f2c176-fd11-48ae-8737-6067672064d6-kube-api-access-hg7kp\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:36.731010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.730979 2561 generic.go:356] "Generic (PLEG): container finished" podID="61f2c176-fd11-48ae-8737-6067672064d6" containerID="e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8" exitCode=0 Jul 07 19:45:36.731192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.731031 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" Jul 07 19:45:36.731192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.731029 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" event={"ID":"61f2c176-fd11-48ae-8737-6067672064d6","Type":"ContainerDied","Data":"e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8"} Jul 07 19:45:36.731192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.731178 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp" event={"ID":"61f2c176-fd11-48ae-8737-6067672064d6","Type":"ContainerDied","Data":"56adf5145b89ed048be5121212ca047599b8d5f5f26e5c017d308a137aa75f2c"} Jul 07 19:45:36.731313 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.731218 2561 scope.go:117] "RemoveContainer" containerID="e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8" Jul 07 19:45:36.739950 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.739927 2561 scope.go:117] "RemoveContainer" containerID="e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8" Jul 07 19:45:36.740212 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:36.740191 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8\": container with ID starting with e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8 not found: ID does not exist" containerID="e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8" Jul 07 19:45:36.740295 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.740223 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8"} err="failed to get container status \"e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8\": rpc error: code = NotFound desc = could not find container \"e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8\": container with ID starting with e3b993cd42fc4703257457b08919a652bdba665e6241a74a5f2125d08c92fff8 not found: ID does not exist" Jul 07 19:45:36.757475 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.757455 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp"] Jul 07 19:45:36.760934 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:36.760915 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-be0d7986-openshift-default-57c9c46dc7-fp5cp"] Jul 07 19:45:37.292313 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.292283 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh"] Jul 07 19:45:37.292835 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.292817 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61f2c176-fd11-48ae-8737-6067672064d6" containerName="istio-proxy" Jul 07 19:45:37.292910 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.292839 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="61f2c176-fd11-48ae-8737-6067672064d6" containerName="istio-proxy" Jul 07 19:45:37.292910 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.292883 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="093324ec-b860-4d33-b0a1-d06676a624cf" containerName="maas-api" Jul 07 19:45:37.292910 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.292892 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="093324ec-b860-4d33-b0a1-d06676a624cf" containerName="maas-api" Jul 07 19:45:37.293060 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.292986 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="093324ec-b860-4d33-b0a1-d06676a624cf" containerName="maas-api" Jul 07 19:45:37.293060 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.293001 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="61f2c176-fd11-48ae-8737-6067672064d6" containerName="istio-proxy" Jul 07 19:45:37.296666 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.296455 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.299114 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.299093 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d70e5b04-openshift-default-dockercfg-rtxzq\"" Jul 07 19:45:37.312072 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.312051 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh"] Jul 07 19:45:37.345246 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345217 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-envoy\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istiod-ca-cert\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345302 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqm4w\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-kube-api-access-sqm4w\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345326 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-certs\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345349 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-credential-socket\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345609 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-data\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345609 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345469 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-socket\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345609 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345484 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-podinfo\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.345609 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.345501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-token\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.446747 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-data\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.446930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446754 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-socket\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.446930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-podinfo\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.446930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-token\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.446930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-envoy\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.446930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istiod-ca-cert\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446934 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqm4w\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-kube-api-access-sqm4w\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.446975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-certs\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.447004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-credential-socket\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.447177 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-socket\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.447197 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-data\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.447430 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-certs\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447766 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.447552 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-credential-socket\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.447766 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.447641 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istiod-ca-cert\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.449654 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.449627 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-envoy\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.449843 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.449828 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-podinfo\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.455459 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.455436 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-token\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.455562 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.455526 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqm4w\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-kube-api-access-sqm4w\") pod \"e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.607628 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.607532 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:37.743175 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.743153 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh"] Jul 07 19:45:37.745311 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:37.745284 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c14a37a_0add_42cc_bb4e_31cb5543f9ee.slice/crio-76579a353bde42508d0efdfb054b0ed3815f1a9c1bfc101e4e2c6e1d2578c429 WatchSource:0}: Error finding container 76579a353bde42508d0efdfb054b0ed3815f1a9c1bfc101e4e2c6e1d2578c429: Status 404 returned error can't find the container with id 76579a353bde42508d0efdfb054b0ed3815f1a9c1bfc101e4e2c6e1d2578c429 Jul 07 19:45:37.747351 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.747311 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:37.747471 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.747408 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:37.747471 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.747440 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:37.802262 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.802236 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:37.902987 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:37.902904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:38.449017 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.448986 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:38.508952 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.508920 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77"] Jul 07 19:45:38.511794 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.511770 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.514988 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.514786 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-d70e5b04-serving-cert\"" Jul 07 19:45:38.514988 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.514810 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:45:38.532963 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.532940 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77"] Jul 07 19:45:38.550904 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.550876 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:38.565560 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.565537 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsdpr\" (UniqueName: \"kubernetes.io/projected/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-kube-api-access-wsdpr\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.565667 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.565600 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.565667 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.565629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-openshift-service-ca\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.627951 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.626499 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61f2c176-fd11-48ae-8737-6067672064d6" path="/var/lib/kubelet/pods/61f2c176-fd11-48ae-8737-6067672064d6/volumes" Jul 07 19:45:38.657449 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.657419 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:38.666401 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.666366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.666551 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.666421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-openshift-service-ca\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.666551 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.666461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsdpr\" (UniqueName: \"kubernetes.io/projected/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-kube-api-access-wsdpr\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.666551 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:38.666488 2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-d70e5b04-serving-cert: secret "maas-api-e2e-ait-d70e5b04-serving-cert" not found Jul 07 19:45:38.666551 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:38.666551 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls podName:8be083c7-64ac-45e0-99a8-d44b7f3bb0a3 nodeName:}" failed. No retries permitted until 2026-07-07 19:45:39.166533154 +0000 UTC m=+1485.068411690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls") pod "maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" (UID: "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3") : secret "maas-api-e2e-ait-d70e5b04-serving-cert" not found Jul 07 19:45:38.667158 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.667137 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-openshift-service-ca\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.681545 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.681516 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsdpr\" (UniqueName: \"kubernetes.io/projected/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-kube-api-access-wsdpr\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:38.743086 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.743008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" event={"ID":"3c14a37a-0add-42cc-bb4e-31cb5543f9ee","Type":"ContainerStarted","Data":"8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33"} Jul 07 19:45:38.743086 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.743067 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" event={"ID":"3c14a37a-0add-42cc-bb4e-31cb5543f9ee","Type":"ContainerStarted","Data":"76579a353bde42508d0efdfb054b0ed3815f1a9c1bfc101e4e2c6e1d2578c429"} Jul 07 19:45:38.785750 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.785693 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" podStartSLOduration=1.785673474 podStartE2EDuration="1.785673474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:38.784852482 +0000 UTC m=+1484.686731018" watchObservedRunningTime="2026-07-07 19:45:38.785673474 +0000 UTC m=+1484.687552026" Jul 07 19:45:38.841719 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:38.841690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:39.170801 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.170718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:39.173124 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.173094 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls\") pod \"maas-api-e2e-ait-d70e5b04-b8476d6-g5w77\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:39.210493 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.210470 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:39.311535 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.311485 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:39.425445 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.425339 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:39.562079 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.562002 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77"] Jul 07 19:45:39.564306 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:39.564269 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be083c7_64ac_45e0_99a8_d44b7f3bb0a3.slice/crio-72321b4f8917501cf5ebd8eba2136dd075c3882d6b78b298cc7603519c66a636 WatchSource:0}: Error finding container 72321b4f8917501cf5ebd8eba2136dd075c3882d6b78b298cc7603519c66a636: Status 404 returned error can't find the container with id 72321b4f8917501cf5ebd8eba2136dd075c3882d6b78b298cc7603519c66a636 Jul 07 19:45:39.608514 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.608485 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:39.614080 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.614040 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:39.749938 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.749904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" event={"ID":"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3","Type":"ContainerStarted","Data":"72321b4f8917501cf5ebd8eba2136dd075c3882d6b78b298cc7603519c66a636"} Jul 07 19:45:39.750129 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.750109 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:39.750990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:39.750974 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:40.571150 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:40.571119 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:40.756567 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:40.756474 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" event={"ID":"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3","Type":"ContainerStarted","Data":"1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4"} Jul 07 19:45:40.756782 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:40.756764 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:40.776180 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:40.776129 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" podStartSLOduration=2.465642865 podStartE2EDuration="2.776114301s" totalImagesPullingTime="310.471436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:40.775662287 +0000 UTC m=+1486.677540835" watchObservedRunningTime="2026-07-07 19:45:40.776114301 +0000 UTC m=+1486.677992836" Jul 07 19:45:41.202369 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:41.202335 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:45:43.597835 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:43.597809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:43.957675 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:43.957648 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77"] Jul 07 19:45:43.958099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:43.958045 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" podUID="8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" containerName="maas-api" containerID="cri-o://1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4" gracePeriod=30 Jul 07 19:45:43.989532 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:43.989503 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:44.250172 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.250146 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:44.340500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.340464 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsdpr\" (UniqueName: \"kubernetes.io/projected/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-kube-api-access-wsdpr\") pod \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " Jul 07 19:45:44.340686 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.340577 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls\") pod \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " Jul 07 19:45:44.340758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.340683 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-openshift-service-ca\") pod \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\" (UID: \"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3\") " Jul 07 19:45:44.341293 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.341270 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" (UID: "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:45:44.343149 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.343119 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" (UID: "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:45:44.343401 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.343346 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-kube-api-access-wsdpr" (OuterVolumeSpecName: "kube-api-access-wsdpr") pod "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" (UID: "8be083c7-64ac-45e0-99a8-d44b7f3bb0a3"). InnerVolumeSpecName "kube-api-access-wsdpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:44.442154 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.442114 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsdpr\" (UniqueName: \"kubernetes.io/projected/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-kube-api-access-wsdpr\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:44.442154 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.442156 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-maas-api-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:44.442344 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.442172 2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3-openshift-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:44.777704 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.777673 2561 generic.go:356] "Generic (PLEG): container finished" podID="8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" containerID="1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4" exitCode=0 Jul 07 19:45:44.778157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.777719 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" Jul 07 19:45:44.778157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.777758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" event={"ID":"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3","Type":"ContainerDied","Data":"1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4"} Jul 07 19:45:44.778157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.777811 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77" event={"ID":"8be083c7-64ac-45e0-99a8-d44b7f3bb0a3","Type":"ContainerDied","Data":"72321b4f8917501cf5ebd8eba2136dd075c3882d6b78b298cc7603519c66a636"} Jul 07 19:45:44.778157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.777836 2561 scope.go:117] "RemoveContainer" containerID="1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4" Jul 07 19:45:44.786571 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.786556 2561 scope.go:117] "RemoveContainer" containerID="1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4" Jul 07 19:45:44.786831 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:44.786813 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4\": container with ID starting with 1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4 not found: ID does not exist" containerID="1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4" Jul 07 19:45:44.786890 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.786842 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4"} err="failed to get container status \"1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4\": rpc error: code = NotFound desc = could not find container \"1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4\": container with ID starting with 1c5062a37ad1a36f7734dd30401461eb22134921783951c3af234638597ed2b4 not found: ID does not exist" Jul 07 19:45:44.798044 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.798020 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77"] Jul 07 19:45:44.800346 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:44.800328 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-d70e5b04-b8476d6-g5w77"] Jul 07 19:45:46.623909 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:46.623875 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" path="/var/lib/kubelet/pods/8be083c7-64ac-45e0-99a8-d44b7f3bb0a3/volumes" Jul 07 19:45:48.802543 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:48.802517 2561 generic.go:356] "Generic (PLEG): container finished" podID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerID="8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33" exitCode=0 Jul 07 19:45:48.802948 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:48.802594 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" event={"ID":"3c14a37a-0add-42cc-bb4e-31cb5543f9ee","Type":"ContainerDied","Data":"8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33"} Jul 07 19:45:48.802991 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:48.802960 2561 scope.go:117] "RemoveContainer" containerID="8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33" Jul 07 19:45:48.803909 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:48.803881 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:48.804002 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:48.803969 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:48.804122 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:48.804019 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:49.670237 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:49.670206 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh"] Jul 07 19:45:49.704060 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:49.704028 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:49.809579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:49.809542 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" event={"ID":"3c14a37a-0add-42cc-bb4e-31cb5543f9ee","Type":"ContainerStarted","Data":"4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6"} Jul 07 19:45:49.809972 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:49.809632 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" containerID="cri-o://4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6" gracePeriod=30 Jul 07 19:45:49.820877 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:49.820851 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:51.484750 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:51.484718 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:52.123265 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:52.123231 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:45:54.627950 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:54.627921 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:45:54.633199 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:54.633175 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:45:54.750321 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:54.750293 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:55.070244 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.070221 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:55.234128 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234099 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-socket\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234158 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istiod-ca-cert\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234177 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-data\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234194 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-token\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234227 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-envoy\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234243 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-podinfo\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234264 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-certs\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234310 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234293 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqm4w\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-kube-api-access-sqm4w\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234660 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234320 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-credential-socket\") pod \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\" (UID: \"3c14a37a-0add-42cc-bb4e-31cb5543f9ee\") " Jul 07 19:45:55.234660 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234444 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:55.234660 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234636 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.234815 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234741 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:55.234881 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234836 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:45:55.235005 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.234978 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:55.235137 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.235083 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:55.236671 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.236648 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:55.236671 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.236656 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 19:45:55.236822 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.236702 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-kube-api-access-sqm4w" (OuterVolumeSpecName: "kube-api-access-sqm4w") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "kube-api-access-sqm4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:45:55.236822 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.236720 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c14a37a-0add-42cc-bb4e-31cb5543f9ee" (UID: "3c14a37a-0add-42cc-bb4e-31cb5543f9ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:45:55.335350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335319 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istiod-ca-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335346 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-data\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335355 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335363 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-envoy\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335371 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-istio-podinfo\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335402 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-workload-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335410 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqm4w\" (UniqueName: \"kubernetes.io/projected/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-kube-api-access-sqm4w\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.335623 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.335419 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c14a37a-0add-42cc-bb4e-31cb5543f9ee-credential-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:45:55.845487 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.845456 2561 generic.go:356] "Generic (PLEG): container finished" podID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerID="4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6" exitCode=0 Jul 07 19:45:55.845896 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.845503 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" Jul 07 19:45:55.845896 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.845535 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" event={"ID":"3c14a37a-0add-42cc-bb4e-31cb5543f9ee","Type":"ContainerDied","Data":"4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6"} Jul 07 19:45:55.845896 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.845589 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh" event={"ID":"3c14a37a-0add-42cc-bb4e-31cb5543f9ee","Type":"ContainerDied","Data":"76579a353bde42508d0efdfb054b0ed3815f1a9c1bfc101e4e2c6e1d2578c429"} Jul 07 19:45:55.845896 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.845623 2561 scope.go:117] "RemoveContainer" containerID="4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6" Jul 07 19:45:55.854586 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.854565 2561 scope.go:117] "RemoveContainer" containerID="8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33" Jul 07 19:45:55.862328 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.862312 2561 scope.go:117] "RemoveContainer" containerID="4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6" Jul 07 19:45:55.862574 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:55.862554 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6\": container with ID starting with 4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6 not found: ID does not exist" containerID="4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6" Jul 07 19:45:55.862641 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.862582 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6"} err="failed to get container status \"4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6\": rpc error: code = NotFound desc = could not find container \"4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6\": container with ID starting with 4ba1ada8a6cc398f33ea85e332ab8779da25a3adc402c29bdec5c1171bad81b6 not found: ID does not exist" Jul 07 19:45:55.862641 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.862598 2561 scope.go:117] "RemoveContainer" containerID="8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33" Jul 07 19:45:55.862834 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:55.862820 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33\": container with ID starting with 8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33 not found: ID does not exist" containerID="8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33" Jul 07 19:45:55.862873 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.862840 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33"} err="failed to get container status \"8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33\": rpc error: code = NotFound desc = could not find container \"8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33\": container with ID starting with 8677069646c36de35143190d8b4a4ea533f31a5045e480b9966f3b2284d28e33 not found: ID does not exist" Jul 07 19:45:55.872306 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.872275 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh"] Jul 07 19:45:55.874012 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.873995 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d70e5b04-openshift-default-6b4f679bcc-847gh"] Jul 07 19:45:55.876965 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.876949 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:55.948892 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:55.948861 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:56.052534 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052499 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj"] Jul 07 19:45:56.052904 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052891 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" Jul 07 19:45:56.052945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052906 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" Jul 07 19:45:56.052945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052915 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" containerName="maas-api" Jul 07 19:45:56.052945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052921 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" containerName="maas-api" Jul 07 19:45:56.052945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052929 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" Jul 07 19:45:56.052945 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.052939 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" Jul 07 19:45:56.053102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.053029 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" Jul 07 19:45:56.053102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.053040 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be083c7-64ac-45e0-99a8-d44b7f3bb0a3" containerName="maas-api" Jul 07 19:45:56.056193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.056170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.059341 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.059321 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2b04b691-openshift-default-dockercfg-d6wkf\"" Jul 07 19:45:56.069468 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.069446 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj"] Jul 07 19:45:56.089478 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.089457 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:56.143253 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-credential-socket\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143253 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143203 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istiod-ca-cert\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143253 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slsfb\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-kube-api-access-slsfb\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-envoy\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143325 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-data\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143343 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-socket\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143358 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-certs\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143504 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-token\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.143663 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.143500 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-podinfo\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.189374 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.189335 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:56.244725 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244690 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-data\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244725 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244730 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-socket\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-certs\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244777 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-token\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-podinfo\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-credential-socket\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istiod-ca-cert\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.244976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244937 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slsfb\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-kube-api-access-slsfb\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.245692 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.244989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-envoy\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.245692 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.245428 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-certs\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.245692 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.245603 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-data\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.245692 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.245679 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istiod-ca-cert\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.245838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.245751 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-socket\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.245838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.245767 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-credential-socket\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.247529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.247509 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-podinfo\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.247529 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.247524 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-envoy\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.253640 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.253620 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-token\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.253741 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.253721 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slsfb\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-kube-api-access-slsfb\") pod \"e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.367321 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.367283 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:56.505493 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.505470 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj"] Jul 07 19:45:56.507740 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:56.507714 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3e1f5b_0025_4802_bbe0_14679a6142ab.slice/crio-ed38b6a0f49becd6164ee29e29567eb50746118672d0d1b2ac42fe97c79c1526 WatchSource:0}: Error finding container ed38b6a0f49becd6164ee29e29567eb50746118672d0d1b2ac42fe97c79c1526: Status 404 returned error can't find the container with id ed38b6a0f49becd6164ee29e29567eb50746118672d0d1b2ac42fe97c79c1526 Jul 07 19:45:56.509907 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.509880 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:56.509976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.509936 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:56.509976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.509965 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:45:56.625392 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.625356 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" path="/var/lib/kubelet/pods/3c14a37a-0add-42cc-bb4e-31cb5543f9ee/volumes" Jul 07 19:45:56.852285 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.852247 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" event={"ID":"ea3e1f5b-0025-4802-bbe0-14679a6142ab","Type":"ContainerStarted","Data":"2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36"} Jul 07 19:45:56.852755 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.852309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" event={"ID":"ea3e1f5b-0025-4802-bbe0-14679a6142ab","Type":"ContainerStarted","Data":"ed38b6a0f49becd6164ee29e29567eb50746118672d0d1b2ac42fe97c79c1526"} Jul 07 19:45:56.872140 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:56.872087 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" podStartSLOduration=0.87206103 podStartE2EDuration="872.06103ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:56.871832418 +0000 UTC m=+1502.773710975" watchObservedRunningTime="2026-07-07 19:45:56.87206103 +0000 UTC m=+1502.773939566" Jul 07 19:45:57.280652 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.280619 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:57.368428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.368400 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:57.373308 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.373286 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:57.390482 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.390460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:57.436302 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.436263 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772"] Jul 07 19:45:57.436734 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.436718 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c14a37a-0add-42cc-bb4e-31cb5543f9ee" containerName="istio-proxy" Jul 07 19:45:57.441935 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.441914 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.444998 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.444964 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-2b04b691-serving-cert\"" Jul 07 19:45:57.445207 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.445186 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:45:57.457355 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.457334 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772"] Jul 07 19:45:57.556527 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.556444 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.556679 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.556588 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283e6e32-fc5a-453c-a306-4fce63f8f7da-openshift-service-ca\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.556679 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.556631 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68zxp\" (UniqueName: \"kubernetes.io/projected/283e6e32-fc5a-453c-a306-4fce63f8f7da-kube-api-access-68zxp\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.582016 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.581988 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:57.657328 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.657297 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283e6e32-fc5a-453c-a306-4fce63f8f7da-openshift-service-ca\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.657492 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.657337 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68zxp\" (UniqueName: \"kubernetes.io/projected/283e6e32-fc5a-453c-a306-4fce63f8f7da-kube-api-access-68zxp\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.657492 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.657369 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.657584 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:57.657538 2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-2b04b691-serving-cert: secret "maas-api-e2e-derive-2b04b691-serving-cert" not found Jul 07 19:45:57.657628 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:45:57.657613 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls podName:283e6e32-fc5a-453c-a306-4fce63f8f7da nodeName:}" failed. No retries permitted until 2026-07-07 19:45:58.157593521 +0000 UTC m=+1504.059472047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls") pod "maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" (UID: "283e6e32-fc5a-453c-a306-4fce63f8f7da") : secret "maas-api-e2e-derive-2b04b691-serving-cert" not found Jul 07 19:45:57.658075 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.658054 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283e6e32-fc5a-453c-a306-4fce63f8f7da-openshift-service-ca\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.670210 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.670179 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68zxp\" (UniqueName: \"kubernetes.io/projected/283e6e32-fc5a-453c-a306-4fce63f8f7da-kube-api-access-68zxp\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:57.832095 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.832017 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:57.858129 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.858101 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:57.859134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:57.859115 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:45:58.147659 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.147593 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:58.161797 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.161774 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:58.164000 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.163980 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls\") pod \"maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:58.255597 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.255569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:58.354556 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.354526 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:58.488993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.488898 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772"] Jul 07 19:45:58.491880 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:45:58.491849 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283e6e32_fc5a_453c_a306_4fce63f8f7da.slice/crio-a3b5a2ae0750ed691dbc769cc103a5c42181909da06264dd58ecbc16d35796ab WatchSource:0}: Error finding container a3b5a2ae0750ed691dbc769cc103a5c42181909da06264dd58ecbc16d35796ab: Status 404 returned error can't find the container with id a3b5a2ae0750ed691dbc769cc103a5c42181909da06264dd58ecbc16d35796ab Jul 07 19:45:58.863986 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:58.863958 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" event={"ID":"283e6e32-fc5a-453c-a306-4fce63f8f7da","Type":"ContainerStarted","Data":"a3b5a2ae0750ed691dbc769cc103a5c42181909da06264dd58ecbc16d35796ab"} Jul 07 19:45:59.485835 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:59.485801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:45:59.870724 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:59.870626 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" event={"ID":"283e6e32-fc5a-453c-a306-4fce63f8f7da","Type":"ContainerStarted","Data":"146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017"} Jul 07 19:45:59.871174 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:59.870893 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:45:59.890214 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:45:59.890169 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" podStartSLOduration=2.586364665 podStartE2EDuration="2.890156421s" totalImagesPullingTime="303.791756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:45:59.888369344 +0000 UTC m=+1505.790247880" watchObservedRunningTime="2026-07-07 19:45:59.890156421 +0000 UTC m=+1505.792034955" Jul 07 19:46:00.478941 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:00.478912 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:46:01.696685 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:01.696642 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:01.819011 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:01.818972 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj"] Jul 07 19:46:01.819509 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:01.819451 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" podUID="ea3e1f5b-0025-4802-bbe0-14679a6142ab" containerName="istio-proxy" containerID="cri-o://2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36" gracePeriod=30 Jul 07 19:46:01.842305 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:01.842270 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:01.942831 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:01.942791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:02.065692 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.065614 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772"] Jul 07 19:46:02.065866 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.065810 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" podUID="283e6e32-fc5a-453c-a306-4fce63f8f7da" containerName="maas-api" containerID="cri-o://146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017" gracePeriod=30 Jul 07 19:46:02.106641 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.106617 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:02.363136 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.363106 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:46:02.501626 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.501593 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283e6e32-fc5a-453c-a306-4fce63f8f7da-openshift-service-ca\") pod \"283e6e32-fc5a-453c-a306-4fce63f8f7da\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " Jul 07 19:46:02.501803 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.501787 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls\") pod \"283e6e32-fc5a-453c-a306-4fce63f8f7da\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " Jul 07 19:46:02.501873 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.501839 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68zxp\" (UniqueName: \"kubernetes.io/projected/283e6e32-fc5a-453c-a306-4fce63f8f7da-kube-api-access-68zxp\") pod \"283e6e32-fc5a-453c-a306-4fce63f8f7da\" (UID: \"283e6e32-fc5a-453c-a306-4fce63f8f7da\") " Jul 07 19:46:02.501929 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.501892 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/283e6e32-fc5a-453c-a306-4fce63f8f7da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "283e6e32-fc5a-453c-a306-4fce63f8f7da" (UID: "283e6e32-fc5a-453c-a306-4fce63f8f7da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:46:02.502083 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.502069 2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/283e6e32-fc5a-453c-a306-4fce63f8f7da-openshift-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:02.503764 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.503735 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283e6e32-fc5a-453c-a306-4fce63f8f7da-kube-api-access-68zxp" (OuterVolumeSpecName: "kube-api-access-68zxp") pod "283e6e32-fc5a-453c-a306-4fce63f8f7da" (UID: "283e6e32-fc5a-453c-a306-4fce63f8f7da"). InnerVolumeSpecName "kube-api-access-68zxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:46:02.503764 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.503760 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "283e6e32-fc5a-453c-a306-4fce63f8f7da" (UID: "283e6e32-fc5a-453c-a306-4fce63f8f7da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:46:02.603007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.602923 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68zxp\" (UniqueName: \"kubernetes.io/projected/283e6e32-fc5a-453c-a306-4fce63f8f7da-kube-api-access-68zxp\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:02.603007 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.602954 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/283e6e32-fc5a-453c-a306-4fce63f8f7da-maas-api-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:02.886989 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.886900 2561 generic.go:356] "Generic (PLEG): container finished" podID="283e6e32-fc5a-453c-a306-4fce63f8f7da" containerID="146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017" exitCode=0 Jul 07 19:46:02.886989 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.886943 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" Jul 07 19:46:02.886989 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.886971 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" event={"ID":"283e6e32-fc5a-453c-a306-4fce63f8f7da","Type":"ContainerDied","Data":"146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017"} Jul 07 19:46:02.887531 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.887013 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772" event={"ID":"283e6e32-fc5a-453c-a306-4fce63f8f7da","Type":"ContainerDied","Data":"a3b5a2ae0750ed691dbc769cc103a5c42181909da06264dd58ecbc16d35796ab"} Jul 07 19:46:02.887531 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.887035 2561 scope.go:117] "RemoveContainer" containerID="146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017" Jul 07 19:46:02.895906 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.895890 2561 scope.go:117] "RemoveContainer" containerID="146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017" Jul 07 19:46:02.896169 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:46:02.896150 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017\": container with ID starting with 146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017 not found: ID does not exist" containerID="146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017" Jul 07 19:46:02.896240 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.896181 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017"} err="failed to get container status \"146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017\": rpc error: code = NotFound desc = could not find container \"146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017\": container with ID starting with 146c3de59a16c822c2d6f6bdf2af9234590ecaac3da2347a72bf44558e24e017 not found: ID does not exist" Jul 07 19:46:02.905372 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.905355 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772"] Jul 07 19:46:02.909763 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:02.909744 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-2b04b691-7b795cd5fd-8s772"] Jul 07 19:46:03.554556 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:03.554523 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:04.626898 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:04.626866 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="283e6e32-fc5a-453c-a306-4fce63f8f7da" path="/var/lib/kubelet/pods/283e6e32-fc5a-453c-a306-4fce63f8f7da/volumes" Jul 07 19:46:08.079701 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.079640 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:46:08.202772 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.202746 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:08.256245 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256217 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-certs\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256286 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slsfb\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-kube-api-access-slsfb\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256307 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-podinfo\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256352 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-credential-socket\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256416 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256368 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istiod-ca-cert\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256483 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:46:08.256639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256511 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-data\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256543 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-token\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256605 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-envoy\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256839 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256643 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-socket\") pod \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\" (UID: \"ea3e1f5b-0025-4802-bbe0-14679a6142ab\") " Jul 07 19:46:08.256839 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256672 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:46:08.256839 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.256776 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:46:08.257051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.257031 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-credential-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.257110 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.257056 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istiod-ca-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.257110 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.257064 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:46:08.257110 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.257071 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.257353 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.257324 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:46:08.258863 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.258838 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:46:08.258972 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.258892 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:46:08.259088 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.259067 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-kube-api-access-slsfb" (OuterVolumeSpecName: "kube-api-access-slsfb") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "kube-api-access-slsfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:46:08.259138 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.259109 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea3e1f5b-0025-4802-bbe0-14679a6142ab" (UID: "ea3e1f5b-0025-4802-bbe0-14679a6142ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 19:46:08.299936 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.299912 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:08.358157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.358132 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-data\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.358157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.358151 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.358157 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.358159 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-envoy\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.358323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.358167 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3e1f5b-0025-4802-bbe0-14679a6142ab-workload-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.358323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.358176 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slsfb\" (UniqueName: \"kubernetes.io/projected/ea3e1f5b-0025-4802-bbe0-14679a6142ab-kube-api-access-slsfb\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.358323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.358186 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3e1f5b-0025-4802-bbe0-14679a6142ab-istio-podinfo\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:46:08.693558 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.693507 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:08.797163 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.797123 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:08.894631 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.894547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:08.930992 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.930948 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea3e1f5b-0025-4802-bbe0-14679a6142ab" containerID="2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36" exitCode=0 Jul 07 19:46:08.931184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.931040 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" Jul 07 19:46:08.931184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.931042 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" event={"ID":"ea3e1f5b-0025-4802-bbe0-14679a6142ab","Type":"ContainerDied","Data":"2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36"} Jul 07 19:46:08.931184 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.931157 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj" event={"ID":"ea3e1f5b-0025-4802-bbe0-14679a6142ab","Type":"ContainerDied","Data":"ed38b6a0f49becd6164ee29e29567eb50746118672d0d1b2ac42fe97c79c1526"} Jul 07 19:46:08.931335 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.931197 2561 scope.go:117] "RemoveContainer" containerID="2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36" Jul 07 19:46:08.939914 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.939710 2561 scope.go:117] "RemoveContainer" containerID="2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36" Jul 07 19:46:08.939996 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:46:08.939964 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36\": container with ID starting with 2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36 not found: ID does not exist" containerID="2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36" Jul 07 19:46:08.940048 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.939995 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36"} err="failed to get container status \"2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36\": rpc error: code = NotFound desc = could not find container \"2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36\": container with ID starting with 2114ce437cd8ef0f917c04c97ad2cf42ed1c00b27a99099fdab54bbd692ade36 not found: ID does not exist" Jul 07 19:46:08.952869 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.952847 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj"] Jul 07 19:46:08.964165 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:08.964146 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2b04b691-openshift-default-58968898d7-sh9sj"] Jul 07 19:46:10.283607 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:10.283572 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:10.625148 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:10.625071 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea3e1f5b-0025-4802-bbe0-14679a6142ab" path="/var/lib/kubelet/pods/ea3e1f5b-0025-4802-bbe0-14679a6142ab/volumes" Jul 07 19:46:12.589428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:12.589395 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:46:16.451944 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:16.451867 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:16.604371 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:16.604338 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:16.714991 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:16.714909 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:17.296180 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:17.296145 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:20.451672 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:20.451640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:20.518229 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:20.518194 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:43.179477 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:43.179448 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:49.388646 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:49.388598 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:49.502984 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:49.502953 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:49.673230 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:49.673139 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:52.850359 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:52.850326 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:46:52.950465 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:46:52.950422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:05.510916 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:05.510880 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:11.950317 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:11.950285 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:12.126303 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:12.126268 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:12.299805 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:12.299712 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:22.760829 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:22.760793 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:23.569426 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:23.569370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:30.021976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:30.021945 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:30.128554 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:30.128512 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:30.223423 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:30.223372 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:30.321557 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:30.321472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:31.176151 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:31.176120 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:31.367053 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:31.367024 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:32.582577 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:32.582540 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:47:33.037535 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:33.037503 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:33.538345 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:33.538312 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:39.473443 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:39.473391 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:39.657260 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:39.657231 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:47:39.725231 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:39.725128 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:40.181584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:40.181490 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:46.396770 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.396672 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:46.477770 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.477723 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:46.655047 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.654970 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:46.904656 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.904623 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:47:46.965702 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.965676 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5"] Jul 07 19:47:46.966052 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.966040 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="283e6e32-fc5a-453c-a306-4fce63f8f7da" containerName="maas-api" Jul 07 19:47:46.966102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.966054 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="283e6e32-fc5a-453c-a306-4fce63f8f7da" containerName="maas-api" Jul 07 19:47:46.966102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.966066 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3e1f5b-0025-4802-bbe0-14679a6142ab" containerName="istio-proxy" Jul 07 19:47:46.966102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.966073 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3e1f5b-0025-4802-bbe0-14679a6142ab" containerName="istio-proxy" Jul 07 19:47:46.966193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.966146 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3e1f5b-0025-4802-bbe0-14679a6142ab" containerName="istio-proxy" Jul 07 19:47:46.966193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.966155 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="283e6e32-fc5a-453c-a306-4fce63f8f7da" containerName="maas-api" Jul 07 19:47:46.968547 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.968529 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:46.972673 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:46.972656 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a6d4b797-openshift-default-dockercfg-tr67z\"" Jul 07 19:47:47.002015 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.001982 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5"] Jul 07 19:47:47.010789 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.010717 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:47.110649 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110608 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-socket\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110649 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110653 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-podinfo\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110724 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-envoy\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110749 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-credential-socket\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110777 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-token\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-certs\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110832 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-data\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.110920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p55f\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-kube-api-access-5p55f\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.111238 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.110934 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istiod-ca-cert\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211572 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-socket\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211572 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211573 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-podinfo\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-envoy\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-credential-socket\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211664 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-token\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211687 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-certs\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211715 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-data\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211739 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p55f\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-kube-api-access-5p55f\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.211834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.211768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istiod-ca-cert\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.212193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.212014 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-socket\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.212193 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.212088 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-certs\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.212509 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.212446 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-data\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.212509 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.212499 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-credential-socket\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.212933 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.212906 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istiod-ca-cert\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.214130 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.214114 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-envoy\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.215082 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.215057 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-podinfo\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.227815 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.227745 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p55f\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-kube-api-access-5p55f\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.229556 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.229536 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-token\") pod \"e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.239926 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.239904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:47.280570 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.280548 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:47.429250 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.429227 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5"] Jul 07 19:47:47.430202 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:47:47.430173 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e87548_a014_4f92_a696_3c8d19a6f7ce.slice/crio-d3b7621b65f71d2ebcea366b00bd9b72ad75a46ed0891047207a48e9f8b3ff4c WatchSource:0}: Error finding container d3b7621b65f71d2ebcea366b00bd9b72ad75a46ed0891047207a48e9f8b3ff4c: Status 404 returned error can't find the container with id d3b7621b65f71d2ebcea366b00bd9b72ad75a46ed0891047207a48e9f8b3ff4c Jul 07 19:47:47.432058 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.432023 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:47:47.432122 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.432092 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:47:47.432160 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:47.432126 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:47:48.413617 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:48.413585 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" event={"ID":"f1e87548-a014-4f92-a696-3c8d19a6f7ce","Type":"ContainerStarted","Data":"db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1"} Jul 07 19:47:48.413778 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:48.413632 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" event={"ID":"f1e87548-a014-4f92-a696-3c8d19a6f7ce","Type":"ContainerStarted","Data":"d3b7621b65f71d2ebcea366b00bd9b72ad75a46ed0891047207a48e9f8b3ff4c"} Jul 07 19:47:48.437203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:48.437148 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podStartSLOduration=2.43713115 podStartE2EDuration="2.43713115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:47:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:47:48.43593594 +0000 UTC m=+1614.337814489" watchObservedRunningTime="2026-07-07 19:47:48.43713115 +0000 UTC m=+1614.339009686" Jul 07 19:47:49.281620 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:49.281588 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:49.286500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:49.286473 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:49.419713 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:49.419683 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:49.420577 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:49.420552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:47:50.395476 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.395437 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:50.452076 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.452047 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d"] Jul 07 19:47:50.455588 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.455568 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.458057 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.458034 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-a6d4b797-serving-cert\"" Jul 07 19:47:50.458483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.458455 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:47:50.471452 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.471431 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d"] Jul 07 19:47:50.491175 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.491150 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:50.591867 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.591836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:50.644771 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.644737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.644771 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.644772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz849\" (UniqueName: \"kubernetes.io/projected/e8248844-eddb-4f40-b44f-ac6d48aedfb4-kube-api-access-wz849\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.645010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.644867 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8248844-eddb-4f40-b44f-ac6d48aedfb4-openshift-service-ca\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.746077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.746044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.746077 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.746078 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz849\" (UniqueName: \"kubernetes.io/projected/e8248844-eddb-4f40-b44f-ac6d48aedfb4-kube-api-access-wz849\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.746322 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.746123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8248844-eddb-4f40-b44f-ac6d48aedfb4-openshift-service-ca\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.746322 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:47:50.746190 2561 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-a6d4b797-serving-cert: secret "maas-api-e2e-mt-a6d4b797-serving-cert" not found Jul 07 19:47:50.746322 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:47:50.746261 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls podName:e8248844-eddb-4f40-b44f-ac6d48aedfb4 nodeName:}" failed. No retries permitted until 2026-07-07 19:47:51.246242006 +0000 UTC m=+1617.148120539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls") pod "maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" (UID: "e8248844-eddb-4f40-b44f-ac6d48aedfb4") : secret "maas-api-e2e-mt-a6d4b797-serving-cert" not found Jul 07 19:47:50.746792 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.746775 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8248844-eddb-4f40-b44f-ac6d48aedfb4-openshift-service-ca\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.757187 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.757165 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz849\" (UniqueName: \"kubernetes.io/projected/e8248844-eddb-4f40-b44f-ac6d48aedfb4-kube-api-access-wz849\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:50.903089 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:50.903054 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:51.037629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:51.037538 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:51.137999 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:51.137968 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:51.250534 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:51.250501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:51.252920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:51.252897 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls\") pod \"maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:51.368250 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:51.368161 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:51.505846 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:51.505821 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d"] Jul 07 19:47:51.507648 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:47:51.507617 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8248844_eddb_4f40_b44f_ac6d48aedfb4.slice/crio-87de65f5c5bf6d7e9e6047ce0579f5dd22be8b37a79f68da66e7f4607add22fd WatchSource:0}: Error finding container 87de65f5c5bf6d7e9e6047ce0579f5dd22be8b37a79f68da66e7f4607add22fd: Status 404 returned error can't find the container with id 87de65f5c5bf6d7e9e6047ce0579f5dd22be8b37a79f68da66e7f4607add22fd Jul 07 19:47:52.439620 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:52.439576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" event={"ID":"e8248844-eddb-4f40-b44f-ac6d48aedfb4","Type":"ContainerStarted","Data":"27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37"} Jul 07 19:47:52.439816 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:52.439746 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:52.439816 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:52.439765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" event={"ID":"e8248844-eddb-4f40-b44f-ac6d48aedfb4","Type":"ContainerStarted","Data":"87de65f5c5bf6d7e9e6047ce0579f5dd22be8b37a79f68da66e7f4607add22fd"} Jul 07 19:47:52.457987 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:52.457944 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" podStartSLOduration=2.056315769 podStartE2EDuration="2.457930001s" totalImagesPullingTime="401.614232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:47:52.457366064 +0000 UTC m=+1618.359244621" watchObservedRunningTime="2026-07-07 19:47:52.457930001 +0000 UTC m=+1618.359808535" Jul 07 19:47:54.885867 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:54.885836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:56.977889 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:56.977859 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:47:58.450153 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:58.450117 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:47:59.485100 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:59.485064 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:47:59.551834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:47:59.551802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:48:04.425799 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.425581 2561 patch_prober.go:28] interesting pod/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:59608->10.132.0.49:15021: read: connection reset by peer" start-of-body= Jul 07 19:48:04.425799 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.425640 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:59608->10.132.0.49:15021: read: connection reset by peer" Jul 07 19:48:04.505191 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.505163 2561 generic.go:356] "Generic (PLEG): container finished" podID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerID="db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1" exitCode=0 Jul 07 19:48:04.505329 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.505241 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" event={"ID":"f1e87548-a014-4f92-a696-3c8d19a6f7ce","Type":"ContainerDied","Data":"db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1"} Jul 07 19:48:04.505643 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.505629 2561 scope.go:117] "RemoveContainer" containerID="db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1" Jul 07 19:48:04.506639 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.506612 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:48:04.506718 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.506683 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:48:04.506756 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:04.506725 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:48:05.511851 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:05.511809 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" event={"ID":"f1e87548-a014-4f92-a696-3c8d19a6f7ce","Type":"ContainerStarted","Data":"6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8"} Jul 07 19:48:06.281493 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:06.281461 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:48:06.281696 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:06.281645 2561 patch_prober.go:28] interesting pod/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 19:48:06.281772 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:06.281700 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 19:48:07.280944 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:07.280910 2561 patch_prober.go:28] interesting pod/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 19:48:07.281340 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:07.280963 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 19:48:08.281572 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:08.281534 2561 patch_prober.go:28] interesting pod/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 19:48:08.281961 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:08.281602 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 19:48:09.281827 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:09.281794 2561 patch_prober.go:28] interesting pod/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 07 19:48:09.282210 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:09.281843 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 07 19:48:10.237818 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:10.237786 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:48:10.284335 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:10.284310 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:48:10.284801 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:10.284729 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:48:10.285436 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:10.285419 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:48:10.338880 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:10.338840 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:48:10.440115 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:10.440082 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:48:14.310541 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:14.310512 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:48:47.323200 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:47.323166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:48:47.417791 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:48:47.417757 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:49:54.636112 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:49:54.636084 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:49:55.675729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:49:55.675698 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:50:54.663061 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:54.663033 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:50:54.669989 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:54.669967 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:50:57.841898 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:57.841864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:50:58.239577 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.239546 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:50:58.342437 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.342398 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:50:58.440911 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.440879 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:50:58.578472 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.578367 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d"] Jul 07 19:50:58.578702 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.578670 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" podUID="e8248844-eddb-4f40-b44f-ac6d48aedfb4" containerName="maas-api" containerID="cri-o://27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37" gracePeriod=30 Jul 07 19:50:58.610523 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.610500 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:50:58.833010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.832943 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:50:58.838568 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.838545 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8248844-eddb-4f40-b44f-ac6d48aedfb4-openshift-service-ca\") pod \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " Jul 07 19:50:58.838720 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.838622 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls\") pod \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " Jul 07 19:50:58.838720 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.838649 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz849\" (UniqueName: \"kubernetes.io/projected/e8248844-eddb-4f40-b44f-ac6d48aedfb4-kube-api-access-wz849\") pod \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\" (UID: \"e8248844-eddb-4f40-b44f-ac6d48aedfb4\") " Jul 07 19:50:58.838937 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.838915 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8248844-eddb-4f40-b44f-ac6d48aedfb4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8248844-eddb-4f40-b44f-ac6d48aedfb4" (UID: "e8248844-eddb-4f40-b44f-ac6d48aedfb4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:50:58.840797 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.840770 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8248844-eddb-4f40-b44f-ac6d48aedfb4-kube-api-access-wz849" (OuterVolumeSpecName: "kube-api-access-wz849") pod "e8248844-eddb-4f40-b44f-ac6d48aedfb4" (UID: "e8248844-eddb-4f40-b44f-ac6d48aedfb4"). InnerVolumeSpecName "kube-api-access-wz849". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:50:58.841097 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.841069 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8248844-eddb-4f40-b44f-ac6d48aedfb4" (UID: "e8248844-eddb-4f40-b44f-ac6d48aedfb4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:50:58.939588 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.939554 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz849\" (UniqueName: \"kubernetes.io/projected/e8248844-eddb-4f40-b44f-ac6d48aedfb4-kube-api-access-wz849\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:50:58.939588 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.939584 2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8248844-eddb-4f40-b44f-ac6d48aedfb4-openshift-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:50:58.939588 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:58.939595 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8248844-eddb-4f40-b44f-ac6d48aedfb4-maas-api-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:50:59.365850 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.365817 2561 generic.go:356] "Generic (PLEG): container finished" podID="e8248844-eddb-4f40-b44f-ac6d48aedfb4" containerID="27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37" exitCode=0 Jul 07 19:50:59.366020 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.365884 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" Jul 07 19:50:59.366020 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.365904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" event={"ID":"e8248844-eddb-4f40-b44f-ac6d48aedfb4","Type":"ContainerDied","Data":"27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37"} Jul 07 19:50:59.366020 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.365990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d" event={"ID":"e8248844-eddb-4f40-b44f-ac6d48aedfb4","Type":"ContainerDied","Data":"87de65f5c5bf6d7e9e6047ce0579f5dd22be8b37a79f68da66e7f4607add22fd"} Jul 07 19:50:59.366020 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.366019 2561 scope.go:117] "RemoveContainer" containerID="27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37" Jul 07 19:50:59.375449 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.375433 2561 scope.go:117] "RemoveContainer" containerID="27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37" Jul 07 19:50:59.375685 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:50:59.375666 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37\": container with ID starting with 27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37 not found: ID does not exist" containerID="27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37" Jul 07 19:50:59.375755 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.375696 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37"} err="failed to get container status \"27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37\": rpc error: code = NotFound desc = could not find container \"27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37\": container with ID starting with 27c43bd1b8e95401202b797c31321bc49f96efa113dd229bfd3ffd95d1566d37 not found: ID does not exist" Jul 07 19:50:59.389161 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.389137 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d"] Jul 07 19:50:59.394074 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:50:59.394056 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-a6d4b797-748f5646d6-9nf6d"] Jul 07 19:51:00.623690 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:00.623661 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8248844-eddb-4f40-b44f-ac6d48aedfb4" path="/var/lib/kubelet/pods/e8248844-eddb-4f40-b44f-ac6d48aedfb4/volumes" Jul 07 19:51:01.177160 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:01.177129 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:51:04.412871 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:04.412830 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:04.509292 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:04.509256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:04.659645 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:04.659613 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5"] Jul 07 19:51:04.659880 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:04.659855 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" containerID="cri-o://6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8" gracePeriod=30 Jul 07 19:51:04.684492 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:04.684457 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:04.924857 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:04.924824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:05.116103 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:05.116010 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:05.214591 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:05.214533 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:05.352867 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:05.352837 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:09.915404 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.915364 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:51:09.934350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934332 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-token\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934466 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934363 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-certs\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934466 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934418 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-credential-socket\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934466 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934444 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p55f\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-kube-api-access-5p55f\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934595 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934474 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istiod-ca-cert\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934595 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934531 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-data\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934735 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934709 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-envoy\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934814 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934782 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:51:09.934814 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934787 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:51:09.934924 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934813 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-socket\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.934924 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934863 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-podinfo\") pod \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\" (UID: \"f1e87548-a014-4f92-a696-3c8d19a6f7ce\") " Jul 07 19:51:09.935016 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.934965 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:51:09.935084 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.935055 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:51:09.935265 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.935184 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:09.935265 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.935200 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-workload-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:09.935265 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.935209 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-credential-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:09.935265 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.935217 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istiod-ca-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:09.935676 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.935651 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:51:09.936629 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.936608 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:51:09.937010 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.936991 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 19:51:09.937069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.937032 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:51:09.937069 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:09.937049 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-kube-api-access-5p55f" (OuterVolumeSpecName: "kube-api-access-5p55f") pod "f1e87548-a014-4f92-a696-3c8d19a6f7ce" (UID: "f1e87548-a014-4f92-a696-3c8d19a6f7ce"). InnerVolumeSpecName "kube-api-access-5p55f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:51:10.035707 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.035682 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:10.035707 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.035707 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p55f\" (UniqueName: \"kubernetes.io/projected/f1e87548-a014-4f92-a696-3c8d19a6f7ce-kube-api-access-5p55f\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:10.035935 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.035719 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-data\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:10.035935 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.035731 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-envoy\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:10.035935 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.035742 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1e87548-a014-4f92-a696-3c8d19a6f7ce-istio-podinfo\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:51:10.435833 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.435808 2561 generic.go:356] "Generic (PLEG): container finished" podID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerID="6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8" exitCode=0 Jul 07 19:51:10.435990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.435859 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" Jul 07 19:51:10.435990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.435889 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" event={"ID":"f1e87548-a014-4f92-a696-3c8d19a6f7ce","Type":"ContainerDied","Data":"6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8"} Jul 07 19:51:10.435990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.435947 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5" event={"ID":"f1e87548-a014-4f92-a696-3c8d19a6f7ce","Type":"ContainerDied","Data":"d3b7621b65f71d2ebcea366b00bd9b72ad75a46ed0891047207a48e9f8b3ff4c"} Jul 07 19:51:10.435990 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.435977 2561 scope.go:117] "RemoveContainer" containerID="6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8" Jul 07 19:51:10.445761 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.445743 2561 scope.go:117] "RemoveContainer" containerID="db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1" Jul 07 19:51:10.453470 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.453453 2561 scope.go:117] "RemoveContainer" containerID="6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8" Jul 07 19:51:10.453713 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:51:10.453695 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8\": container with ID starting with 6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8 not found: ID does not exist" containerID="6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8" Jul 07 19:51:10.453759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.453718 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8"} err="failed to get container status \"6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8\": rpc error: code = NotFound desc = could not find container \"6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8\": container with ID starting with 6486d08c0e3554ec9b067fdbcadf11ba0751c8ad7ac8c04ba84f9f3d832da4e8 not found: ID does not exist" Jul 07 19:51:10.453759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.453733 2561 scope.go:117] "RemoveContainer" containerID="db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1" Jul 07 19:51:10.453933 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:51:10.453914 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1\": container with ID starting with db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1 not found: ID does not exist" containerID="db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1" Jul 07 19:51:10.453978 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.453939 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1"} err="failed to get container status \"db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1\": rpc error: code = NotFound desc = could not find container \"db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1\": container with ID starting with db3343653eebb6786449309dcdbf0a5cccd8f2c8ae51e4a7c28ef6859c0e70e1 not found: ID does not exist" Jul 07 19:51:10.459266 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.459248 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5"] Jul 07 19:51:10.467907 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.467886 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a6d4b797-openshift-default-5d7f86d96c-6zcc5"] Jul 07 19:51:10.624285 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:10.624252 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" path="/var/lib/kubelet/pods/f1e87548-a014-4f92-a696-3c8d19a6f7ce/volumes" Jul 07 19:51:11.440707 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:11.440673 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:11.554683 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:11.554653 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:11.639414 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:11.639363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:11.806501 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:11.806412 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:18.594370 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:18.594337 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:18.659117 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:18.659082 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:25.498737 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.498705 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:25.870097 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870024 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj"] Jul 07 19:51:25.870407 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870369 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" Jul 07 19:51:25.870458 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870409 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" Jul 07 19:51:25.870458 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870426 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8248844-eddb-4f40-b44f-ac6d48aedfb4" containerName="maas-api" Jul 07 19:51:25.870458 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870431 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8248844-eddb-4f40-b44f-ac6d48aedfb4" containerName="maas-api" Jul 07 19:51:25.870562 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870505 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" Jul 07 19:51:25.870562 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870515 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" Jul 07 19:51:25.870562 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.870522 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8248844-eddb-4f40-b44f-ac6d48aedfb4" containerName="maas-api" Jul 07 19:51:25.874164 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.874145 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.874316 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.874253 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:25.877130 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.877113 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c9eca85c-openshift-default-dockercfg-fz78n\"" Jul 07 19:51:25.896402 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.896366 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj"] Jul 07 19:51:25.974145 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974111 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-certs\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974145 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-data\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-token\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-podinfo\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974405 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvhc\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-kube-api-access-jpvhc\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974610 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974451 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-socket\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974610 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-envoy\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974610 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-credential-socket\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:25.974610 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:25.974567 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istiod-ca-cert\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075394 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-podinfo\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvhc\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-kube-api-access-jpvhc\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-socket\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075480 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-envoy\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075505 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-credential-socket\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istiod-ca-cert\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075807 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075567 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-certs\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075807 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075594 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-data\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075807 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-token\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.075967 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.075908 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-credential-socket\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.076026 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.076002 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-socket\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.076086 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.076049 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-certs\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.076149 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.076085 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-data\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.076496 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.076473 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istiod-ca-cert\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.077701 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.077672 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-envoy\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.077823 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.077807 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-podinfo\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.088719 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.088696 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-token\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.089323 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.089303 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvhc\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-kube-api-access-jpvhc\") pod \"e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.099285 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.099263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:26.184287 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.184266 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:26.323976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.323943 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj"] Jul 07 19:51:26.326202 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:51:26.326179 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50735f28_4f3e_48eb_a80c_f6bfb913e0ce.slice/crio-6eb1cff435bea2e0ae7a252531a2099401382c3602bcd07d4044847b99c24207 WatchSource:0}: Error finding container 6eb1cff435bea2e0ae7a252531a2099401382c3602bcd07d4044847b99c24207: Status 404 returned error can't find the container with id 6eb1cff435bea2e0ae7a252531a2099401382c3602bcd07d4044847b99c24207 Jul 07 19:51:26.328235 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.328202 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:51:26.328331 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.328309 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:51:26.328372 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.328356 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:51:26.514520 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.514423 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" event={"ID":"50735f28-4f3e-48eb-a80c-f6bfb913e0ce","Type":"ContainerStarted","Data":"94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c"} Jul 07 19:51:26.514520 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.514479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" event={"ID":"50735f28-4f3e-48eb-a80c-f6bfb913e0ce","Type":"ContainerStarted","Data":"6eb1cff435bea2e0ae7a252531a2099401382c3602bcd07d4044847b99c24207"} Jul 07 19:51:26.538315 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:26.538260 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" podStartSLOduration=1.5382396219999999 podStartE2EDuration="1.538239622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:51:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:51:26.535943561 +0000 UTC m=+1832.437822108" watchObservedRunningTime="2026-07-07 19:51:26.538239622 +0000 UTC m=+1832.440118158" Jul 07 19:51:27.185006 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:27.184978 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:27.189590 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:27.189570 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:27.520428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:27.520339 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:27.521272 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:27.521253 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:28.482739 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.482699 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:28.585993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.585964 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:28.591962 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.591931 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp"] Jul 07 19:51:28.592571 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.592550 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" Jul 07 19:51:28.592625 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.592576 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1e87548-a014-4f92-a696-3c8d19a6f7ce" containerName="istio-proxy" Jul 07 19:51:28.595137 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.595117 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.597960 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.597937 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-c9eca85c-serving-cert\"" Jul 07 19:51:28.598065 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.597982 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:51:28.609321 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.609301 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp"] Jul 07 19:51:28.679645 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.679612 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:28.699837 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.699794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgv7f\" (UniqueName: \"kubernetes.io/projected/f18cf335-fd9b-4046-aa8a-df50a74494c8-kube-api-access-hgv7f\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.700061 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.699855 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f18cf335-fd9b-4046-aa8a-df50a74494c8-openshift-service-ca\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.700123 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.700086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f18cf335-fd9b-4046-aa8a-df50a74494c8-maas-api-tls\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.780848 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.780736 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:28.801426 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.801396 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgv7f\" (UniqueName: \"kubernetes.io/projected/f18cf335-fd9b-4046-aa8a-df50a74494c8-kube-api-access-hgv7f\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.801576 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.801454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f18cf335-fd9b-4046-aa8a-df50a74494c8-openshift-service-ca\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.801576 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.801521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f18cf335-fd9b-4046-aa8a-df50a74494c8-maas-api-tls\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.802203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.802176 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f18cf335-fd9b-4046-aa8a-df50a74494c8-openshift-service-ca\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.804051 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.804024 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f18cf335-fd9b-4046-aa8a-df50a74494c8-maas-api-tls\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.810634 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.810607 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgv7f\" (UniqueName: \"kubernetes.io/projected/f18cf335-fd9b-4046-aa8a-df50a74494c8-kube-api-access-hgv7f\") pod \"maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:28.906320 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:28.906288 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:29.043788 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:29.043763 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp"] Jul 07 19:51:29.045616 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:51:29.045593 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf18cf335_fd9b_4046_aa8a_df50a74494c8.slice/crio-cebeafe7b77e2f4ff7e3b7f1cc01e68de5fc546961519ed397a7ad7a81450b65 WatchSource:0}: Error finding container cebeafe7b77e2f4ff7e3b7f1cc01e68de5fc546961519ed397a7ad7a81450b65: Status 404 returned error can't find the container with id cebeafe7b77e2f4ff7e3b7f1cc01e68de5fc546961519ed397a7ad7a81450b65 Jul 07 19:51:29.047452 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:29.047435 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:51:29.187539 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:29.187509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:29.288530 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:29.288487 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:29.530518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:29.530491 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" event={"ID":"f18cf335-fd9b-4046-aa8a-df50a74494c8","Type":"ContainerStarted","Data":"cebeafe7b77e2f4ff7e3b7f1cc01e68de5fc546961519ed397a7ad7a81450b65"} Jul 07 19:51:30.536753 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:30.536719 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" event={"ID":"f18cf335-fd9b-4046-aa8a-df50a74494c8","Type":"ContainerStarted","Data":"d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85"} Jul 07 19:51:30.537154 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:30.536784 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:30.556971 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:30.556902 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" podStartSLOduration=2.086698292 podStartE2EDuration="2.556876622s" totalImagesPullingTime="470.17833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:51:30.553498821 +0000 UTC m=+1836.455377381" watchObservedRunningTime="2026-07-07 19:51:30.556876622 +0000 UTC m=+1836.458755158" Jul 07 19:51:30.879574 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:30.879489 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:51:32.961500 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:32.961468 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:51:36.548691 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:36.548661 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:51:38.579552 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:38.579520 2561 generic.go:356] "Generic (PLEG): container finished" podID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerID="94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c" exitCode=0 Jul 07 19:51:38.579917 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:38.579594 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" event={"ID":"50735f28-4f3e-48eb-a80c-f6bfb913e0ce","Type":"ContainerDied","Data":"94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c"} Jul 07 19:51:38.580005 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:38.579992 2561 scope.go:117] "RemoveContainer" containerID="94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c" Jul 07 19:51:38.580974 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:38.580946 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:51:38.581081 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:38.581029 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:51:38.581161 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:38.581078 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:51:39.588847 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:39.588811 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" event={"ID":"50735f28-4f3e-48eb-a80c-f6bfb913e0ce","Type":"ContainerStarted","Data":"b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9"} Jul 07 19:51:40.185015 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:40.184980 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:40.186033 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:40.186012 2561 patch_prober.go:28] interesting pod/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 19:51:40.186098 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:40.186062 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 19:51:41.185397 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:41.185333 2561 patch_prober.go:28] interesting pod/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 19:51:41.185832 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:41.185434 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 19:51:42.184809 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:42.184772 2561 patch_prober.go:28] interesting pod/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 19:51:42.184993 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:42.184841 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 19:51:42.521260 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:42.521154 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:43.185092 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:43.185061 2561 patch_prober.go:28] interesting pod/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 07 19:51:43.185331 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:43.185305 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 07 19:51:44.189036 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:44.189000 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:51:44.190067 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:51:44.190049 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:52:01.744409 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:52:01.744360 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:53:14.528027 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:53:14.527991 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:53:48.189919 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:53:48.189830 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:53:48.290280 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:53:48.290247 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:37.926666 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:37.926631 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:38.358102 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:38.358070 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:38.585792 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:38.585763 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:38.697927 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:38.697899 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:38.926371 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:38.926342 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp"] Jul 07 19:54:38.926585 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:38.926562 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" podUID="f18cf335-fd9b-4046-aa8a-df50a74494c8" containerName="maas-api" containerID="cri-o://d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85" gracePeriod=30 Jul 07 19:54:39.031714 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.031628 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:39.171357 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.171333 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:54:39.218756 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.218723 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f18cf335-fd9b-4046-aa8a-df50a74494c8-openshift-service-ca\") pod \"f18cf335-fd9b-4046-aa8a-df50a74494c8\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " Jul 07 19:54:39.218935 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.218824 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f18cf335-fd9b-4046-aa8a-df50a74494c8-maas-api-tls\") pod \"f18cf335-fd9b-4046-aa8a-df50a74494c8\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " Jul 07 19:54:39.218935 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.218875 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgv7f\" (UniqueName: \"kubernetes.io/projected/f18cf335-fd9b-4046-aa8a-df50a74494c8-kube-api-access-hgv7f\") pod \"f18cf335-fd9b-4046-aa8a-df50a74494c8\" (UID: \"f18cf335-fd9b-4046-aa8a-df50a74494c8\") " Jul 07 19:54:39.219159 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.219137 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f18cf335-fd9b-4046-aa8a-df50a74494c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f18cf335-fd9b-4046-aa8a-df50a74494c8" (UID: "f18cf335-fd9b-4046-aa8a-df50a74494c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:54:39.221023 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.220991 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f18cf335-fd9b-4046-aa8a-df50a74494c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f18cf335-fd9b-4046-aa8a-df50a74494c8" (UID: "f18cf335-fd9b-4046-aa8a-df50a74494c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:54:39.221155 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.221076 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f18cf335-fd9b-4046-aa8a-df50a74494c8-kube-api-access-hgv7f" (OuterVolumeSpecName: "kube-api-access-hgv7f") pod "f18cf335-fd9b-4046-aa8a-df50a74494c8" (UID: "f18cf335-fd9b-4046-aa8a-df50a74494c8"). InnerVolumeSpecName "kube-api-access-hgv7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:54:39.319789 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.319711 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f18cf335-fd9b-4046-aa8a-df50a74494c8-maas-api-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:39.319789 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.319738 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgv7f\" (UniqueName: \"kubernetes.io/projected/f18cf335-fd9b-4046-aa8a-df50a74494c8-kube-api-access-hgv7f\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:39.319789 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.319748 2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f18cf335-fd9b-4046-aa8a-df50a74494c8-openshift-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:39.471239 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.471202 2561 generic.go:356] "Generic (PLEG): container finished" podID="f18cf335-fd9b-4046-aa8a-df50a74494c8" containerID="d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85" exitCode=0 Jul 07 19:54:39.471457 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.471255 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" Jul 07 19:54:39.471457 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.471287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" event={"ID":"f18cf335-fd9b-4046-aa8a-df50a74494c8","Type":"ContainerDied","Data":"d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85"} Jul 07 19:54:39.471457 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.471341 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp" event={"ID":"f18cf335-fd9b-4046-aa8a-df50a74494c8","Type":"ContainerDied","Data":"cebeafe7b77e2f4ff7e3b7f1cc01e68de5fc546961519ed397a7ad7a81450b65"} Jul 07 19:54:39.471457 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.471364 2561 scope.go:117] "RemoveContainer" containerID="d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85" Jul 07 19:54:39.481676 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.481659 2561 scope.go:117] "RemoveContainer" containerID="d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85" Jul 07 19:54:39.481917 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:54:39.481898 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85\": container with ID starting with d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85 not found: ID does not exist" containerID="d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85" Jul 07 19:54:39.481983 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.481931 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85"} err="failed to get container status \"d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85\": rpc error: code = NotFound desc = could not find container \"d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85\": container with ID starting with d64d0b23ddf2a7a236fe500e751afbf9cc802ac1322f80fb069adb00d76ecb85 not found: ID does not exist" Jul 07 19:54:39.505697 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.505668 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp"] Jul 07 19:54:39.510586 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:39.510565 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-c9eca85c-55c844fcb9-r85mp"] Jul 07 19:54:40.624387 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:40.624343 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f18cf335-fd9b-4046-aa8a-df50a74494c8" path="/var/lib/kubelet/pods/f18cf335-fd9b-4046-aa8a-df50a74494c8/volumes" Jul 07 19:54:41.791270 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:41.791234 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:54:44.949507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:44.949469 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:45.057347 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:45.057314 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:45.210887 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:45.210801 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj"] Jul 07 19:54:45.211050 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:45.210987 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" containerID="cri-o://b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9" gracePeriod=30 Jul 07 19:54:45.252399 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:45.252360 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:45.489582 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:45.489497 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:46.679174 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:46.679141 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:46.790390 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:46.790349 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:46.881518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:46.881486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:47.694303 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:47.694267 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:47.812330 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:47.812298 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:47.942214 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:47.942182 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:49.329743 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:49.329690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:50.466049 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.466027 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:54:50.529523 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.529441 2561 generic.go:356] "Generic (PLEG): container finished" podID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerID="b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9" exitCode=0 Jul 07 19:54:50.529523 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.529496 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" Jul 07 19:54:50.529697 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.529496 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" event={"ID":"50735f28-4f3e-48eb-a80c-f6bfb913e0ce","Type":"ContainerDied","Data":"b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9"} Jul 07 19:54:50.529697 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.529615 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj" event={"ID":"50735f28-4f3e-48eb-a80c-f6bfb913e0ce","Type":"ContainerDied","Data":"6eb1cff435bea2e0ae7a252531a2099401382c3602bcd07d4044847b99c24207"} Jul 07 19:54:50.529697 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.529645 2561 scope.go:117] "RemoveContainer" containerID="b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9" Jul 07 19:54:50.541326 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.541306 2561 scope.go:117] "RemoveContainer" containerID="94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c" Jul 07 19:54:50.549696 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.549674 2561 scope.go:117] "RemoveContainer" containerID="b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9" Jul 07 19:54:50.549973 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:54:50.549955 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9\": container with ID starting with b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9 not found: ID does not exist" containerID="b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9" Jul 07 19:54:50.550032 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.549983 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9"} err="failed to get container status \"b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9\": rpc error: code = NotFound desc = could not find container \"b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9\": container with ID starting with b47e18644ac3dcbac5f534bae9bae35a34d27321f0dd3e7e2d035d75d61e34d9 not found: ID does not exist" Jul 07 19:54:50.550032 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.550003 2561 scope.go:117] "RemoveContainer" containerID="94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c" Jul 07 19:54:50.550251 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:54:50.550234 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c\": container with ID starting with 94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c not found: ID does not exist" containerID="94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c" Jul 07 19:54:50.550293 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.550257 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c"} err="failed to get container status \"94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c\": rpc error: code = NotFound desc = could not find container \"94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c\": container with ID starting with 94520cae80f9e2d8bb01c4b1a16eafb583680196df875d39e6c2e8eab7944f5c not found: ID does not exist" Jul 07 19:54:50.555924 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.555902 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:54:50.624276 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624250 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-envoy\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624482 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624295 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpvhc\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-kube-api-access-jpvhc\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624482 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624342 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-credential-socket\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624482 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624413 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-data\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624482 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624460 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istiod-ca-cert\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624489 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-token\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624519 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-socket\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624556 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-certs\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624602 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-podinfo\") pod \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\" (UID: \"50735f28-4f3e-48eb-a80c-f6bfb913e0ce\") " Jul 07 19:54:50.624727 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.624670 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:54:50.625518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.625002 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-credential-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.625518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.625122 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:54:50.625518 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.625393 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:54:50.625735 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.625546 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:54:50.625735 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.625677 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:54:50.627016 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.626988 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-kube-api-access-jpvhc" (OuterVolumeSpecName: "kube-api-access-jpvhc") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "kube-api-access-jpvhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:54:50.627125 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.627036 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:54:50.627174 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.627126 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 19:54:50.627506 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.627486 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "50735f28-4f3e-48eb-a80c-f6bfb913e0ce" (UID: "50735f28-4f3e-48eb-a80c-f6bfb913e0ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:54:50.725601 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725562 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpvhc\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-kube-api-access-jpvhc\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725601 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725597 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-data\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725611 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istiod-ca-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725623 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725636 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725648 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-workload-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725660 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-podinfo\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.725830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.725671 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50735f28-4f3e-48eb-a80c-f6bfb913e0ce-istio-envoy\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:54:50.863216 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.863189 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj"] Jul 07 19:54:50.867329 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:50.867306 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c9eca85c-openshift-default-6cc76dcf47-m6rqj"] Jul 07 19:54:52.623269 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:52.623238 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" path="/var/lib/kubelet/pods/50735f28-4f3e-48eb-a80c-f6bfb913e0ce/volumes" Jul 07 19:54:55.971832 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:55.971797 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:54:57.167732 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:54:57.167690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:00.886079 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:00.886042 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:55:02.834709 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:02.834673 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:02.942363 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:02.942336 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:03.091735 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:03.091654 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:13.581846 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:13.581794 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:13.959674 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:13.959640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:35.045084 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:35.044976 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:41.913508 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:41.913473 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:42.120247 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.120203 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:42.304628 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.304596 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:42.647192 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647106 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt"] Jul 07 19:55:42.647584 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647567 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" Jul 07 19:55:42.647682 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647590 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" Jul 07 19:55:42.647682 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647608 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f18cf335-fd9b-4046-aa8a-df50a74494c8" containerName="maas-api" Jul 07 19:55:42.647682 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647617 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f18cf335-fd9b-4046-aa8a-df50a74494c8" containerName="maas-api" Jul 07 19:55:42.647853 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647720 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" Jul 07 19:55:42.647853 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.647734 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f18cf335-fd9b-4046-aa8a-df50a74494c8" containerName="maas-api" Jul 07 19:55:42.649911 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.649891 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.653360 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.653342 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-00fc51-openshift-default-dockercfg-vzt9w\"" Jul 07 19:55:42.673484 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.673456 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt"] Jul 07 19:55:42.689618 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.689597 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:42.780976 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.780942 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f1379c-158a-4e08-9409-b929a9f08a6e-istiod-ca-cert\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781171 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.780997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-credential-socket\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781171 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-envoy\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781171 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781103 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-podinfo\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781316 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcbgg\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-kube-api-access-bcbgg\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781316 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781221 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-data\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781316 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781274 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-certs\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781318 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-socket\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.781483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.781341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-token\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.882531 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-data\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.882736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882542 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-certs\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.882736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882573 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-socket\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.882736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882620 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-token\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.882736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f1379c-158a-4e08-9409-b929a9f08a6e-istiod-ca-cert\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.882736 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-credential-socket\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883014 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-envoy\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883014 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882907 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-socket\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883014 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-podinfo\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883014 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.882955 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-data\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883214 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.883039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcbgg\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-kube-api-access-bcbgg\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883411 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.883371 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f1379c-158a-4e08-9409-b929a9f08a6e-istiod-ca-cert\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883493 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.883374 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-credential-socket\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.883493 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.883447 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-certs\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.885210 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.885186 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-envoy\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.885405 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.885364 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-podinfo\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.891912 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.891888 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-token\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.892015 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.891953 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcbgg\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-kube-api-access-bcbgg\") pod \"e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:42.960216 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:42.960183 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:43.075424 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.075393 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:43.115889 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.115856 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt"] Jul 07 19:55:43.119493 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:55:43.119447 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f1379c_158a_4e08_9409_b929a9f08a6e.slice/crio-4f926f7a18b08c0ad3cc1de7b7beefbc74ea7a7213d56f029e90d644b3cf2ead WatchSource:0}: Error finding container 4f926f7a18b08c0ad3cc1de7b7beefbc74ea7a7213d56f029e90d644b3cf2ead: Status 404 returned error can't find the container with id 4f926f7a18b08c0ad3cc1de7b7beefbc74ea7a7213d56f029e90d644b3cf2ead Jul 07 19:55:43.122083 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.122038 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:55:43.122185 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.122107 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:55:43.122185 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.122135 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:55:43.793445 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.793410 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" event={"ID":"79f1379c-158a-4e08-9409-b929a9f08a6e","Type":"ContainerStarted","Data":"523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6"} Jul 07 19:55:43.793685 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.793466 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" event={"ID":"79f1379c-158a-4e08-9409-b929a9f08a6e","Type":"ContainerStarted","Data":"4f926f7a18b08c0ad3cc1de7b7beefbc74ea7a7213d56f029e90d644b3cf2ead"} Jul 07 19:55:43.823099 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.823053 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podStartSLOduration=1.823036649 podStartE2EDuration="1.823036649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:55:43.820559774 +0000 UTC m=+2089.722438332" watchObservedRunningTime="2026-07-07 19:55:43.823036649 +0000 UTC m=+2089.724915185" Jul 07 19:55:43.961108 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:43.961077 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:44.965562 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:44.965529 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:45.804691 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.804654 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:45.805515 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.805497 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:55:45.886753 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.886724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:45.953793 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.953759 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh"] Jul 07 19:55:45.954368 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.954343 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" Jul 07 19:55:45.954368 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.954364 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" Jul 07 19:55:45.954551 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.954508 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="50735f28-4f3e-48eb-a80c-f6bfb913e0ce" containerName="istio-proxy" Jul 07 19:55:45.957123 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.957103 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:45.960073 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.960032 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-00fc51-serving-cert\"" Jul 07 19:55:45.960073 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.960060 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k4qkx\"" Jul 07 19:55:45.980759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:45.980736 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh"] Jul 07 19:55:46.021597 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.021563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:46.111123 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.111025 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0242873-de9b-4b9f-a69e-86687c591424-openshift-service-ca\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.111123 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.111064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jbrs\" (UniqueName: \"kubernetes.io/projected/d0242873-de9b-4b9f-a69e-86687c591424-kube-api-access-2jbrs\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.111326 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.111158 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0242873-de9b-4b9f-a69e-86687c591424-maas-api-tls\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.125957 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.125929 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:46.212170 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.212117 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0242873-de9b-4b9f-a69e-86687c591424-maas-api-tls\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.212339 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.212237 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0242873-de9b-4b9f-a69e-86687c591424-openshift-service-ca\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.212339 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.212260 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jbrs\" (UniqueName: \"kubernetes.io/projected/d0242873-de9b-4b9f-a69e-86687c591424-kube-api-access-2jbrs\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.213111 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.213084 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0242873-de9b-4b9f-a69e-86687c591424-openshift-service-ca\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.214731 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.214710 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0242873-de9b-4b9f-a69e-86687c591424-maas-api-tls\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.231473 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.231450 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jbrs\" (UniqueName: \"kubernetes.io/projected/d0242873-de9b-4b9f-a69e-86687c591424-kube-api-access-2jbrs\") pod \"maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.268483 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.268460 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:46.423221 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.423183 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh"] Jul 07 19:55:46.424243 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:55:46.424217 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0242873_de9b_4b9f_a69e_86687c591424.slice/crio-40c8da8d408252ced2544bcb46d665c90e2d2bd0f04f38ad68375f8ac19598b9 WatchSource:0}: Error finding container 40c8da8d408252ced2544bcb46d665c90e2d2bd0f04f38ad68375f8ac19598b9: Status 404 returned error can't find the container with id 40c8da8d408252ced2544bcb46d665c90e2d2bd0f04f38ad68375f8ac19598b9 Jul 07 19:55:46.691020 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.690992 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:46.788300 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.788263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:46.810354 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:46.810325 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" event={"ID":"d0242873-de9b-4b9f-a69e-86687c591424","Type":"ContainerStarted","Data":"40c8da8d408252ced2544bcb46d665c90e2d2bd0f04f38ad68375f8ac19598b9"} Jul 07 19:55:47.818865 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:47.818812 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" event={"ID":"d0242873-de9b-4b9f-a69e-86687c591424","Type":"ContainerStarted","Data":"8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8"} Jul 07 19:55:47.819402 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:47.819168 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:47.862998 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:47.862952 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" podStartSLOduration=2.1441003849999998 podStartE2EDuration="2.86293809s" totalImagesPullingTime="718.837705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:55:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:55:47.862797517 +0000 UTC m=+2093.764676053" watchObservedRunningTime="2026-07-07 19:55:47.86293809 +0000 UTC m=+2093.764816624" Jul 07 19:55:50.397612 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:50.397581 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:55:52.199334 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:52.199301 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:55:53.839141 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:53.839101 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:55:54.691612 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:54.691587 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:55:54.698486 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:55:54.698463 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:56:00.806169 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.806074 2561 patch_prober.go:28] interesting pod/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 07 19:56:00.806810 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.806171 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 07 19:56:00.885983 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.885951 2561 generic.go:356] "Generic (PLEG): container finished" podID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerID="523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6" exitCode=0 Jul 07 19:56:00.886151 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.885989 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" event={"ID":"79f1379c-158a-4e08-9409-b929a9f08a6e","Type":"ContainerDied","Data":"523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6"} Jul 07 19:56:00.886398 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.886365 2561 scope.go:117] "RemoveContainer" containerID="523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6" Jul 07 19:56:00.887428 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.887399 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:56:00.887507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.887468 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:56:00.887507 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:00.887502 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 19:56:01.893490 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:01.893459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" event={"ID":"79f1379c-158a-4e08-9409-b929a9f08a6e","Type":"ContainerStarted","Data":"0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f"} Jul 07 19:56:01.960686 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:01.960654 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:56:01.960950 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:01.960924 2561 patch_prober.go:28] interesting pod/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 07 19:56:01.961060 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:01.960972 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 07 19:56:02.961484 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:02.961447 2561 patch_prober.go:28] interesting pod/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 07 19:56:02.961878 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:02.961506 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 07 19:56:03.960677 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:03.960642 2561 patch_prober.go:28] interesting pod/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 07 19:56:03.960854 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:03.960698 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 07 19:56:04.961075 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:04.961038 2561 patch_prober.go:28] interesting pod/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 07 19:56:04.961536 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:04.961096 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 07 19:56:05.964486 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:05.964456 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:56:05.964963 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:05.964797 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:56:05.965564 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:56:05.965549 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:58:25.096578 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:25.096498 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:49.010754 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:49.010719 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:49.113034 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:49.112972 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:55.263759 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:55.263724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:55.669694 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:55.669610 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:55.770346 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:55.770313 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:55.872216 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:55.872183 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:56.003159 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.003128 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh"] Jul 07 19:58:56.003355 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.003328 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" podUID="d0242873-de9b-4b9f-a69e-86687c591424" containerName="maas-api" containerID="cri-o://8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8" gracePeriod=30 Jul 07 19:58:56.027043 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.027016 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:58:56.259296 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.259237 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:58:56.400581 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.400552 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0242873-de9b-4b9f-a69e-86687c591424-maas-api-tls\") pod \"d0242873-de9b-4b9f-a69e-86687c591424\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " Jul 07 19:58:56.400955 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.400595 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jbrs\" (UniqueName: \"kubernetes.io/projected/d0242873-de9b-4b9f-a69e-86687c591424-kube-api-access-2jbrs\") pod \"d0242873-de9b-4b9f-a69e-86687c591424\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " Jul 07 19:58:56.400955 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.400671 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0242873-de9b-4b9f-a69e-86687c591424-openshift-service-ca\") pod \"d0242873-de9b-4b9f-a69e-86687c591424\" (UID: \"d0242873-de9b-4b9f-a69e-86687c591424\") " Jul 07 19:58:56.401056 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.401036 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0242873-de9b-4b9f-a69e-86687c591424-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0242873-de9b-4b9f-a69e-86687c591424" (UID: "d0242873-de9b-4b9f-a69e-86687c591424"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:58:56.402658 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.402636 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0242873-de9b-4b9f-a69e-86687c591424-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0242873-de9b-4b9f-a69e-86687c591424" (UID: "d0242873-de9b-4b9f-a69e-86687c591424"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:58:56.402757 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.402724 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0242873-de9b-4b9f-a69e-86687c591424-kube-api-access-2jbrs" (OuterVolumeSpecName: "kube-api-access-2jbrs") pod "d0242873-de9b-4b9f-a69e-86687c591424" (UID: "d0242873-de9b-4b9f-a69e-86687c591424"). InnerVolumeSpecName "kube-api-access-2jbrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:58:56.502218 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.502185 2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0242873-de9b-4b9f-a69e-86687c591424-openshift-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:58:56.502218 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.502213 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0242873-de9b-4b9f-a69e-86687c591424-maas-api-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:58:56.502218 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.502222 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jbrs\" (UniqueName: \"kubernetes.io/projected/d0242873-de9b-4b9f-a69e-86687c591424-kube-api-access-2jbrs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:58:56.764621 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.764588 2561 generic.go:356] "Generic (PLEG): container finished" podID="d0242873-de9b-4b9f-a69e-86687c591424" containerID="8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8" exitCode=0 Jul 07 19:58:56.764621 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.764622 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" event={"ID":"d0242873-de9b-4b9f-a69e-86687c591424","Type":"ContainerDied","Data":"8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8"} Jul 07 19:58:56.764838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.764643 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" Jul 07 19:58:56.764838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.764660 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh" event={"ID":"d0242873-de9b-4b9f-a69e-86687c591424","Type":"ContainerDied","Data":"40c8da8d408252ced2544bcb46d665c90e2d2bd0f04f38ad68375f8ac19598b9"} Jul 07 19:58:56.764838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.764681 2561 scope.go:117] "RemoveContainer" containerID="8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8" Jul 07 19:58:56.773230 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.773215 2561 scope.go:117] "RemoveContainer" containerID="8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8" Jul 07 19:58:56.773565 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:58:56.773543 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8\": container with ID starting with 8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8 not found: ID does not exist" containerID="8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8" Jul 07 19:58:56.773669 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.773576 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8"} err="failed to get container status \"8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8\": rpc error: code = NotFound desc = could not find container \"8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8\": container with ID starting with 8935ce03a06a5e81811fc233ee8a14f2319a3ded53230b5372aaae59517764e8 not found: ID does not exist" Jul 07 19:58:56.793448 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.793421 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh"] Jul 07 19:58:56.795109 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:56.795089 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-00fc51-74d7d48f65-77cqh"] Jul 07 19:58:58.624565 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:58.624525 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0242873-de9b-4b9f-a69e-86687c591424" path="/var/lib/kubelet/pods/d0242873-de9b-4b9f-a69e-86687c591424/volumes" Jul 07 19:58:59.077081 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:58:59.077041 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pghc4"] Jul 07 19:59:01.904399 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:01.904351 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:01.979394 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:01.979355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:02.223689 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:02.223658 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt"] Jul 07 19:59:02.223961 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:02.223927 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" containerID="cri-o://0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f" gracePeriod=30 Jul 07 19:59:02.248824 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:02.248796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:02.455554 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:02.455517 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:05.967496 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:05.967461 2561 patch_prober.go:28] interesting pod/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 19:59:05.967866 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:05.967532 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 19:59:07.482459 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.482395 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:59:07.606054 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606021 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-token\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-data\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606123 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-podinfo\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606155 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-certs\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606180 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-credential-socket\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606261 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606253 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f1379c-158a-4e08-9409-b929a9f08a6e-istiod-ca-cert\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606568 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606278 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-socket\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606568 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606367 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcbgg\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-kube-api-access-bcbgg\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606568 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606433 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-envoy\") pod \"79f1379c-158a-4e08-9409-b929a9f08a6e\" (UID: \"79f1379c-158a-4e08-9409-b929a9f08a6e\") " Jul 07 19:59:07.606568 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606489 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:59:07.606568 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606557 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:59:07.606834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606662 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:59:07.606834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606793 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.606834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606811 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-credential-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.606834 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606827 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-workload-socket\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.607061 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606989 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f1379c-158a-4e08-9409-b929a9f08a6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:59:07.607061 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.606994 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:59:07.608487 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.608452 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 19:59:07.608619 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.608572 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-kube-api-access-bcbgg" (OuterVolumeSpecName: "kube-api-access-bcbgg") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "kube-api-access-bcbgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:59:07.608712 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.608697 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:59:07.608779 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.608720 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "79f1379c-158a-4e08-9409-b929a9f08a6e" (UID: "79f1379c-158a-4e08-9409-b929a9f08a6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:59:07.707525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.707491 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcbgg\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-kube-api-access-bcbgg\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.707525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.707521 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-envoy\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.707525 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.707531 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.707760 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.707540 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-data\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.707760 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.707549 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f1379c-158a-4e08-9409-b929a9f08a6e-istio-podinfo\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.707760 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.707557 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f1379c-158a-4e08-9409-b929a9f08a6e-istiod-ca-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 07 19:59:07.831743 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.831656 2561 generic.go:356] "Generic (PLEG): container finished" podID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerID="0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f" exitCode=0 Jul 07 19:59:07.831743 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.831690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" event={"ID":"79f1379c-158a-4e08-9409-b929a9f08a6e","Type":"ContainerDied","Data":"0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f"} Jul 07 19:59:07.831743 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.831703 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" Jul 07 19:59:07.831961 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.831746 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt" event={"ID":"79f1379c-158a-4e08-9409-b929a9f08a6e","Type":"ContainerDied","Data":"4f926f7a18b08c0ad3cc1de7b7beefbc74ea7a7213d56f029e90d644b3cf2ead"} Jul 07 19:59:07.831961 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.831772 2561 scope.go:117] "RemoveContainer" containerID="0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f" Jul 07 19:59:07.841830 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.841814 2561 scope.go:117] "RemoveContainer" containerID="523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6" Jul 07 19:59:07.849605 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.849586 2561 scope.go:117] "RemoveContainer" containerID="0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f" Jul 07 19:59:07.849877 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:59:07.849857 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f\": container with ID starting with 0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f not found: ID does not exist" containerID="0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f" Jul 07 19:59:07.849931 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.849886 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f"} err="failed to get container status \"0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f\": rpc error: code = NotFound desc = could not find container \"0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f\": container with ID starting with 0b0f8042320946def2470e9f3d8d48500e61b246906787d5596315cc136ade8f not found: ID does not exist" Jul 07 19:59:07.849931 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.849902 2561 scope.go:117] "RemoveContainer" containerID="523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6" Jul 07 19:59:07.850103 ip-10-0-130-242 kubenswrapper[2561]: E0707 19:59:07.850087 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6\": container with ID starting with 523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6 not found: ID does not exist" containerID="523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6" Jul 07 19:59:07.850146 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.850106 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6"} err="failed to get container status \"523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6\": rpc error: code = NotFound desc = could not find container \"523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6\": container with ID starting with 523932ed5657e2f5153b8b57ff7f6b73a90309978dfce247c369760817aff5e6 not found: ID does not exist" Jul 07 19:59:07.859853 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.859832 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt"] Jul 07 19:59:07.867658 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:07.867638 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-00fc51-openshift-default-7d6b8474b7-4k7tt"] Jul 07 19:59:08.572765 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:08.572737 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7cgz8_86add30f-218e-45ae-a197-a2f50df4c7ea/manager/0.log" Jul 07 19:59:08.623437 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:08.623405 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" path="/var/lib/kubelet/pods/79f1379c-158a-4e08-9409-b929a9f08a6e/volumes" Jul 07 19:59:08.701559 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:08.701533 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xmvd9_a309fe58-09b9-4a26-96c8-882c6dbea038/manager/0.log" Jul 07 19:59:09.099005 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:09.098977 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724225-w94bv_cbd0e593-2111-4ab0-9454-6c543d548717/cleanup/0.log" Jul 07 19:59:09.699604 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:09.699578 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-kg2c7_68a70b92-cb6b-4112-9bef-1700b07dd50e/copy-manifests/0.log" Jul 07 19:59:09.707920 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:09.707898 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-kg2c7_68a70b92-cb6b-4112-9bef-1700b07dd50e/manager/0.log" Jul 07 19:59:11.399717 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:11.399690 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-56f5l_166367b2-93a7-4900-b08a-c7707545bdf2/manager/0.log" Jul 07 19:59:11.521122 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:11.521098 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2tm9k_14997e14-d133-408b-bc06-9afbb75e84c6/manager/0.log" Jul 07 19:59:11.774576 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:11.774548 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6wz2b_51115018-6adb-4301-b7c7-c157c5a2172d/registry-server/0.log" Jul 07 19:59:11.910647 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:11.910612 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gl7cb_06dbf8c6-8c5b-4381-ac52-2892c679f577/manager/0.log" Jul 07 19:59:12.031131 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:12.031051 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pghc4_910de4bf-d537-4590-8bc6-18da98b5e325/limitador/0.log" Jul 07 19:59:12.523338 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:12.523310 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5xvc_76980d86-780f-4bd2-a924-e588fe09c019/istio-proxy/0.log" Jul 07 19:59:12.884033 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:12.883938 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65ff665485-r72v9_3fab8db2-af52-4a28-a4ad-3fa40b8570da/kube-auth-proxy/0.log" Jul 07 19:59:13.005034 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:13.005009 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/1.log" Jul 07 19:59:13.378908 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:13.378882 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f4d8db47c-k2k2k_00bb5516-a12a-4606-b281-d2cb60b6a0f8/router/0.log" Jul 07 19:59:17.921627 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:17.921596 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:18.902134 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902099 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x8rrx/must-gather-k944q"] Jul 07 19:59:18.902472 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902459 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0242873-de9b-4b9f-a69e-86687c591424" containerName="maas-api" Jul 07 19:59:18.902472 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902472 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0242873-de9b-4b9f-a69e-86687c591424" containerName="maas-api" Jul 07 19:59:18.902579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902488 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" Jul 07 19:59:18.902579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902494 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" Jul 07 19:59:18.902579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902557 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" Jul 07 19:59:18.902579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902565 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" Jul 07 19:59:18.902579 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902572 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0242873-de9b-4b9f-a69e-86687c591424" containerName="maas-api" Jul 07 19:59:18.902780 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902625 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" Jul 07 19:59:18.902780 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.902631 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f1379c-158a-4e08-9409-b929a9f08a6e" containerName="istio-proxy" Jul 07 19:59:18.907418 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.907399 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:18.910139 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.910119 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8rrx\"/\"kube-root-ca.crt\"" Jul 07 19:59:18.910879 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.910857 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x8rrx\"/\"default-dockercfg-65z4s\"" Jul 07 19:59:18.914796 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.914779 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8rrx\"/\"openshift-service-ca.crt\"" Jul 07 19:59:18.916165 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.916141 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:18.920577 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:18.920558 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8rrx/must-gather-k944q"] Jul 07 19:59:19.010135 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.010090 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" limit=1024 Jul 07 19:59:19.012304 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.012279 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45tlh\" (UniqueName: \"kubernetes.io/projected/1588660f-230c-4375-9362-d10eda8e6e95-kube-api-access-45tlh\") pod \"must-gather-k944q\" (UID: \"1588660f-230c-4375-9362-d10eda8e6e95\") " pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.012433 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.012415 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1588660f-230c-4375-9362-d10eda8e6e95-must-gather-output\") pod \"must-gather-k944q\" (UID: \"1588660f-230c-4375-9362-d10eda8e6e95\") " pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.113364 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.113324 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45tlh\" (UniqueName: \"kubernetes.io/projected/1588660f-230c-4375-9362-d10eda8e6e95-kube-api-access-45tlh\") pod \"must-gather-k944q\" (UID: \"1588660f-230c-4375-9362-d10eda8e6e95\") " pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.113576 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.113447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1588660f-230c-4375-9362-d10eda8e6e95-must-gather-output\") pod \"must-gather-k944q\" (UID: \"1588660f-230c-4375-9362-d10eda8e6e95\") " pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.113772 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.113756 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1588660f-230c-4375-9362-d10eda8e6e95-must-gather-output\") pod \"must-gather-k944q\" (UID: \"1588660f-230c-4375-9362-d10eda8e6e95\") " pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.122717 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.122687 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45tlh\" (UniqueName: \"kubernetes.io/projected/1588660f-230c-4375-9362-d10eda8e6e95-kube-api-access-45tlh\") pod \"must-gather-k944q\" (UID: \"1588660f-230c-4375-9362-d10eda8e6e95\") " pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.217144 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.217102 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8rrx/must-gather-k944q" Jul 07 19:59:19.357319 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.357291 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8rrx/must-gather-k944q"] Jul 07 19:59:19.359282 ip-10-0-130-242 kubenswrapper[2561]: W0707 19:59:19.359248 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1588660f_230c_4375_9362_d10eda8e6e95.slice/crio-d9ea9a2091b5b8aa98e3ccb2b4be7d71fdfbd3975b12c4cc2b78f481dd9634b2 WatchSource:0}: Error finding container d9ea9a2091b5b8aa98e3ccb2b4be7d71fdfbd3975b12c4cc2b78f481dd9634b2: Status 404 returned error can't find the container with id d9ea9a2091b5b8aa98e3ccb2b4be7d71fdfbd3975b12c4cc2b78f481dd9634b2 Jul 07 19:59:19.360994 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.360975 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:59:19.891861 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:19.891821 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/must-gather-k944q" event={"ID":"1588660f-230c-4375-9362-d10eda8e6e95","Type":"ContainerStarted","Data":"d9ea9a2091b5b8aa98e3ccb2b4be7d71fdfbd3975b12c4cc2b78f481dd9634b2"} Jul 07 19:59:20.899302 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:20.899257 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/must-gather-k944q" event={"ID":"1588660f-230c-4375-9362-d10eda8e6e95","Type":"ContainerStarted","Data":"0d60c3d68ec8373d537ebd5ddcc5725cc407913652694574a2018844d1c21832"} Jul 07 19:59:20.899755 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:20.899309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/must-gather-k944q" event={"ID":"1588660f-230c-4375-9362-d10eda8e6e95","Type":"ContainerStarted","Data":"9114624dfc0264445731556efdefee6f829389b4e89833b5699b2e4da320b080"} Jul 07 19:59:20.919647 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:20.919582 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x8rrx/must-gather-k944q" podStartSLOduration=1.901092975 podStartE2EDuration="2.919559413s" totalImagesPullingTime="1.018466438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:59:20.917539129 +0000 UTC m=+2306.819417685" watchObservedRunningTime="2026-07-07 19:59:20.919559413 +0000 UTC m=+2306.821437952" Jul 07 19:59:22.085603 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:22.085573 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j474f_b23202de-78de-407f-b354-2c83ecd07eed/global-pull-secret-syncer/0.log" Jul 07 19:59:22.200350 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:22.200320 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d7b7z_6933ad4a-603f-45fc-8821-14edc1cda57e/konnectivity-agent/0.log" Jul 07 19:59:22.316437 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:22.316410 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-242.ec2.internal_e1d6e4c9d780b8824beae118385da5e5/haproxy/0.log" Jul 07 19:59:27.203361 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:27.203325 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-56f5l_166367b2-93a7-4900-b08a-c7707545bdf2/manager/0.log" Jul 07 19:59:27.253769 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:27.253738 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2tm9k_14997e14-d133-408b-bc06-9afbb75e84c6/manager/0.log" Jul 07 19:59:27.340585 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:27.340552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6wz2b_51115018-6adb-4301-b7c7-c157c5a2172d/registry-server/0.log" Jul 07 19:59:27.441982 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:27.441947 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gl7cb_06dbf8c6-8c5b-4381-ac52-2892c679f577/manager/0.log" Jul 07 19:59:27.510713 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:27.510625 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pghc4_910de4bf-d537-4590-8bc6-18da98b5e325/limitador/0.log" Jul 07 19:59:29.516758 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:29.516722 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-26j4c_4983fdb8-9403-40d2-bc8e-18ecd0ee6b74/cluster-monitoring-operator/0.log" Jul 07 19:59:29.830542 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:29.830460 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p74tq_b6650d26-c035-4549-9da7-353999a1c0f2/node-exporter/0.log" Jul 07 19:59:29.860370 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:29.860345 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p74tq_b6650d26-c035-4549-9da7-353999a1c0f2/kube-rbac-proxy/0.log" Jul 07 19:59:29.895449 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:29.895425 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p74tq_b6650d26-c035-4549-9da7-353999a1c0f2/init-textfile/0.log" Jul 07 19:59:31.837877 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.837840 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w"] Jul 07 19:59:31.844439 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.844413 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:31.857671 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.857646 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w"] Jul 07 19:59:31.948052 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.948014 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-podres\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:31.948431 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.948409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg4zk\" (UniqueName: \"kubernetes.io/projected/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-kube-api-access-tg4zk\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:31.948725 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.948672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-lib-modules\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:31.948838 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.948791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-sys\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:31.948904 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.948855 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-proc\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:31.993557 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:31.993517 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-j9bmx_e2e807ca-3d8b-47a9-9b33-bbed5e6fb6cb/networking-console-plugin/0.log" Jul 07 19:59:32.050334 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050287 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-sys\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050603 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-proc\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050603 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-podres\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050603 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050465 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg4zk\" (UniqueName: \"kubernetes.io/projected/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-kube-api-access-tg4zk\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050603 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050524 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-lib-modules\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050864 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050722 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-lib-modules\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050864 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-sys\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.050864 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050836 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-proc\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.051038 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.050916 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-podres\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.062822 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.062793 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg4zk\" (UniqueName: \"kubernetes.io/projected/757d8ad4-a3ed-444d-84cb-6ecc9743e03f-kube-api-access-tg4zk\") pod \"perf-node-gather-daemonset-ncn8w\" (UID: \"757d8ad4-a3ed-444d-84cb-6ecc9743e03f\") " pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.158452 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.158341 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:32.348888 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.348815 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w"] Jul 07 19:59:32.673635 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.673551 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/1.log" Jul 07 19:59:32.681848 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.681809 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tbdsk_ca4bb462-baf6-4e2f-94f8-cb50aa27cbd5/console-operator/2.log" Jul 07 19:59:32.986930 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.986890 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" event={"ID":"757d8ad4-a3ed-444d-84cb-6ecc9743e03f","Type":"ContainerStarted","Data":"3b61ebf74f7d7c73e78894654f5aae6fd66ffd763c71b547c3b10c70faec4570"} Jul 07 19:59:32.987361 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.986990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" event={"ID":"757d8ad4-a3ed-444d-84cb-6ecc9743e03f","Type":"ContainerStarted","Data":"f8bb1d568302800c45377e2853ff25027b01082c06b1c6a8efdcdc85d5943f56"} Jul 07 19:59:32.987361 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:32.987018 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:33.019203 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:33.019132 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" podStartSLOduration=2.019108907 podStartE2EDuration="2.019108907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:59:33.016969066 +0000 UTC m=+2318.918847618" watchObservedRunningTime="2026-07-07 19:59:33.019108907 +0000 UTC m=+2318.920987445" Jul 07 19:59:33.275872 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:33.275767 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-4nbcv_2f618da1-73e6-47a9-9411-c46148b7b563/download-server/0.log" Jul 07 19:59:33.903155 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:33.903130 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-r85f2_5088af8e-6f05-456a-a676-6121c47d54b6/volume-data-source-validator/0.log" Jul 07 19:59:35.090479 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:35.090438 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8bmn_9207323e-9118-4aba-bf0d-47b77fd13358/dns/0.log" Jul 07 19:59:35.116344 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:35.116322 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8bmn_9207323e-9118-4aba-bf0d-47b77fd13358/kube-rbac-proxy/0.log" Jul 07 19:59:35.209941 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:35.209917 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fm4q2_ad798b99-e8b2-439e-ba56-c9805dfacd9d/dns-node-resolver/0.log" Jul 07 19:59:35.909842 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:35.909808 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4w272_0964a1be-fa18-4329-a4c6-e93935334f52/node-ca/0.log" Jul 07 19:59:36.979259 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:36.979230 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5xvc_76980d86-780f-4bd2-a924-e588fe09c019/istio-proxy/0.log" Jul 07 19:59:37.154924 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:37.154888 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65ff665485-r72v9_3fab8db2-af52-4a28-a4ad-3fa40b8570da/kube-auth-proxy/0.log" Jul 07 19:59:37.192289 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:37.192261 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/0.log" Jul 07 19:59:37.227740 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:37.227709 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/1.log" Jul 07 19:59:37.378477 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:37.378404 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f4d8db47c-k2k2k_00bb5516-a12a-4606-b281-d2cb60b6a0f8/router/0.log" Jul 07 19:59:38.067009 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:38.066979 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdbsg_3ecf5971-da77-445e-b564-b403dcdae2ee/serve-healthcheck-canary/0.log" Jul 07 19:59:38.650001 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:38.649968 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7c47h_fb275177-b68f-440a-b440-9fa8d5f4c2c3/insights-operator/0.log" Jul 07 19:59:38.650729 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:38.650713 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7c47h_fb275177-b68f-440a-b440-9fa8d5f4c2c3/insights-operator/1.log" Jul 07 19:59:38.882856 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:38.882832 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t727j_6ea968bb-4c50-4b66-ba6c-cfef43a66b23/kube-rbac-proxy/0.log" Jul 07 19:59:38.915278 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:38.915208 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t727j_6ea968bb-4c50-4b66-ba6c-cfef43a66b23/exporter/0.log" Jul 07 19:59:38.948658 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:38.948628 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t727j_6ea968bb-4c50-4b66-ba6c-cfef43a66b23/extractor/0.log" Jul 07 19:59:39.007622 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:39.007596 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x8rrx/perf-node-gather-daemonset-ncn8w" Jul 07 19:59:41.511908 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:41.511828 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7cgz8_86add30f-218e-45ae-a197-a2f50df4c7ea/manager/0.log" Jul 07 19:59:41.641358 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:41.641265 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xmvd9_a309fe58-09b9-4a26-96c8-882c6dbea038/manager/0.log" Jul 07 19:59:41.755702 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:41.755675 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724225-w94bv_cbd0e593-2111-4ab0-9454-6c543d548717/cleanup/0.log" Jul 07 19:59:42.091739 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.091706 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-kg2c7_68a70b92-cb6b-4112-9bef-1700b07dd50e/manager/0.log" Jul 07 19:59:42.119616 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.119584 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b594cbdb-kg2c7_68a70b92-cb6b-4112-9bef-1700b07dd50e/copy-manifests/0.log" Jul 07 19:59:42.173126 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.173071 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5xvc_76980d86-780f-4bd2-a924-e588fe09c019/istio-proxy/0.log" Jul 07 19:59:42.331805 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.331749 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65ff665485-r72v9_3fab8db2-af52-4a28-a4ad-3fa40b8570da/kube-auth-proxy/0.log" Jul 07 19:59:42.358663 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.358593 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/0.log" Jul 07 19:59:42.392362 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.392337 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/1.log" Jul 07 19:59:42.504564 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:42.504541 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f4d8db47c-k2k2k_00bb5516-a12a-4606-b281-d2cb60b6a0f8/router/0.log" Jul 07 19:59:43.672767 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:43.672743 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5xvc_76980d86-780f-4bd2-a924-e588fe09c019/istio-proxy/0.log" Jul 07 19:59:43.819438 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:43.819414 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65ff665485-r72v9_3fab8db2-af52-4a28-a4ad-3fa40b8570da/kube-auth-proxy/0.log" Jul 07 19:59:43.852534 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:43.852458 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/0.log" Jul 07 19:59:43.886522 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:43.886482 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zcs64_32aa69e5-8cda-4b8f-9ffc-e297e8e80e48/istio-proxy/1.log" Jul 07 19:59:44.004153 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:44.004127 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f4d8db47c-k2k2k_00bb5516-a12a-4606-b281-d2cb60b6a0f8/router/0.log" Jul 07 19:59:48.649510 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:48.649369 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-q68bs_f762579c-08e3-472c-9a89-ca4fe1402c42/migrator/0.log" Jul 07 19:59:48.674877 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:48.674853 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-q68bs_f762579c-08e3-472c-9a89-ca4fe1402c42/graceful-termination/0.log" Jul 07 19:59:49.068259 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:49.068225 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-jdkm8_8f867d34-c4ae-44fb-a644-3faa3118dc5d/kube-storage-version-migrator-operator/1.log" Jul 07 19:59:49.069314 ip-10-0-130-242 kubenswrapper[2561]: I0707 19:59:49.069291 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-jdkm8_8f867d34-c4ae-44fb-a644-3faa3118dc5d/kube-storage-version-migrator-operator/0.log"