Jul 14 17:54:17.009947 ip-10-0-129-115 systemd[1]: Starting Kubernetes Kubelet... Jul 14 17:54:17.458138 ip-10-0-129-115 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:54:17.458138 ip-10-0-129-115 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 17:54:17.458138 ip-10-0-129-115 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:54:17.458138 ip-10-0-129-115 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 17:54:17.458138 ip-10-0-129-115 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:54:17.460276 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.460184 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 17:54:17.465234 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465216 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:54:17.465234 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465234 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:54:17.465234 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465239 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465245 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465249 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465254 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465258 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465262 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465265 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465278 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465282 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465295 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465299 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465302 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465306 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465310 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465314 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465319 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465322 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465326 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465330 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465334 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:54:17.465431 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465338 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465343 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465346 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465351 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465355 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465359 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465363 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465367 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465371 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465375 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465380 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465384 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465389 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465393 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465397 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465401 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465406 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465410 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465415 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465420 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:54:17.466210 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465478 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465525 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465531 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465536 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465541 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465546 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465551 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465555 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465560 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465565 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465569 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465575 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465579 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.465584 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466050 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466065 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466069 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466073 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466076 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466082 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:54:17.466881 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466088 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466092 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466095 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466098 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466101 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466104 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466111 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466117 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466120 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466123 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466128 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466132 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466135 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466138 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466141 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466144 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466146 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466152 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466155 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:54:17.467369 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466158 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466160 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466163 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466165 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466168 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466171 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466173 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466176 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466184 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466187 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466190 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466196 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466199 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466201 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466204 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466207 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466210 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.466212 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467447 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467458 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:54:17.467848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467462 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467465 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467469 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467474 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467478 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467481 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467484 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467487 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467490 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467493 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467495 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467498 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467500 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467503 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467505 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467508 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467510 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467513 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467515 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:54:17.468332 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467518 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467521 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467523 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467525 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467528 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467530 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467533 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467535 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467538 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467540 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467542 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467545 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467550 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467553 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467555 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467557 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467562 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467565 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467568 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:54:17.468820 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467571 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467573 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467576 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467579 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467582 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467584 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467587 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467589 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467592 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467594 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467596 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467600 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467603 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467605 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467623 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467626 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467628 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467631 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467633 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467636 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:54:17.469273 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467638 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467640 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467643 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467646 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467649 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467653 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467656 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467658 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467661 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467663 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467666 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467668 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467671 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467673 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467676 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467678 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467680 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467683 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467685 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467688 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467690 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:54:17.469784 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467692 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467695 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467697 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467700 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467703 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467707 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467709 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467712 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467714 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467717 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467719 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467721 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467724 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467726 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467732 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467735 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467740 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.467748 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468126 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:54:17.470290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468131 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468133 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468136 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468139 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468142 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468145 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468147 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468151 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468153 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468156 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468159 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468161 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468164 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468166 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468169 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468172 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468174 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468176 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468179 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:54:17.470795 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468181 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468184 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468186 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468189 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468191 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468194 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468197 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468201 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468203 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468206 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468209 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468212 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468214 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468217 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468219 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468221 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468224 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468226 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468228 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468231 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:54:17.471249 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468233 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468235 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468238 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468240 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468243 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468245 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468248 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468251 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468253 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468256 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468258 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468260 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468263 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468266 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468268 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468271 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468273 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468275 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468278 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468281 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:54:17.471758 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468283 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468285 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468288 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468291 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468293 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468295 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468298 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468300 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468302 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468305 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468307 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468309 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468312 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468315 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468317 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468320 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468323 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468325 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468327 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:54:17.472264 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468332 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468335 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468338 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468341 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468344 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468349 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468352 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468355 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468357 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468360 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468363 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468366 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468369 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468372 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468375 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468378 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468381 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468383 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468386 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.468388 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:54:17.472747 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468457 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468465 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468470 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468474 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468478 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468481 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468485 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468489 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468492 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468495 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468498 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468502 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468505 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468507 2572 flags.go:64] FLAG: --cgroup-root="" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468510 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468513 2572 flags.go:64] FLAG: --client-ca-file="" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468516 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468519 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468524 2572 flags.go:64] FLAG: --cluster-domain="" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468527 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468530 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468533 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468536 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468540 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 17:54:17.473228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468543 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468546 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468549 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468552 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468555 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468558 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468561 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468563 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468567 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468570 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468573 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468575 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468578 2572 flags.go:64] FLAG: --enable-server="true" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468580 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468584 2572 flags.go:64] FLAG: --event-burst="100" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468587 2572 flags.go:64] FLAG: --event-qps="50" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468590 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468592 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468595 2572 flags.go:64] FLAG: --eviction-hard="" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468598 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468601 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468604 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468621 2572 flags.go:64] FLAG: --eviction-soft="" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468625 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468627 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 17:54:17.473881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468630 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468634 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468636 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468642 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468645 2572 flags.go:64] FLAG: --feature-gates="" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468648 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468651 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468654 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468657 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468661 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468664 2572 flags.go:64] FLAG: --help="false" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468667 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468670 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468673 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468676 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468679 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468682 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468685 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468687 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468690 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468693 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468695 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468698 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468701 2572 flags.go:64] FLAG: --kube-reserved="" Jul 14 17:54:17.474499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468704 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468707 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468709 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468712 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468715 2572 flags.go:64] FLAG: --lock-file="" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468718 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468721 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468723 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468729 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468732 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468734 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468739 2572 flags.go:64] FLAG: --logging-format="text" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468741 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468744 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468747 2572 flags.go:64] FLAG: --manifest-url="" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468750 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468754 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468757 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468762 2572 flags.go:64] FLAG: --max-pods="110" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468765 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468768 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468771 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468774 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468777 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468780 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 17:54:17.475094 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468783 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468789 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468792 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468795 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468798 2572 flags.go:64] FLAG: --pod-cidr="" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468801 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468806 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468809 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468812 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468815 2572 flags.go:64] FLAG: --port="10250" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468818 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468821 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a9c62e5f6ea3f88c" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468824 2572 flags.go:64] FLAG: --qos-reserved="" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468827 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468830 2572 flags.go:64] FLAG: --register-node="true" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468833 2572 flags.go:64] FLAG: --register-with-taints="" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468837 2572 flags.go:64] FLAG: --registry-burst="10" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468839 2572 flags.go:64] FLAG: --registry-qps="5" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468842 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468846 2572 flags.go:64] FLAG: --reserved-memory="" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468850 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468852 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468855 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468858 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468861 2572 flags.go:64] FLAG: --runonce="false" Jul 14 17:54:17.475708 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468864 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468868 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468871 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468873 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468876 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468879 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468882 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468885 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468888 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468890 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468893 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468896 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468899 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468902 2572 flags.go:64] FLAG: --system-cgroups="" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468907 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468912 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468914 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468917 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468922 2572 flags.go:64] FLAG: --tls-min-version="" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468925 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468927 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468930 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468933 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468936 2572 flags.go:64] FLAG: --v="2" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468939 2572 flags.go:64] FLAG: --version="false" Jul 14 17:54:17.476346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468943 2572 flags.go:64] FLAG: --vmodule="" Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468947 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.468951 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469063 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469067 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469070 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469074 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469076 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469079 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469082 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469085 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469088 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469090 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469092 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469095 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469097 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469100 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469102 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469105 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469107 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:54:17.476970 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469110 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469114 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469116 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469119 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469121 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469123 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469126 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469128 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469131 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469133 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469135 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469138 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469140 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469144 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469147 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469149 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469152 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469154 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469156 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:54:17.477506 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469159 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469161 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469164 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469167 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469169 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469172 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469174 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469176 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469179 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469181 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469184 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469186 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469188 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469191 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469194 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469197 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469199 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469202 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469204 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469207 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:54:17.478000 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469209 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469212 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469214 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469216 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469219 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469221 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469225 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469229 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469232 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469235 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469238 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469241 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469243 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469246 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469249 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469251 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469255 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469258 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469260 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:54:17.478495 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469263 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469266 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469268 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469270 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469273 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469275 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469278 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469282 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469285 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469298 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469301 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469304 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469307 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469309 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469312 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469314 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469318 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469322 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469324 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:54:17.478971 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469329 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469332 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469334 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469337 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.469339 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.470176 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.477514 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.477530 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.478424 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.478432 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478489 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478494 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478498 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478501 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478504 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:54:17.479424 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478507 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478510 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478513 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478515 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478518 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478521 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478524 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478526 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478529 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478531 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478534 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478536 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478539 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478541 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478544 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478546 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478548 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478551 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478553 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478556 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:54:17.479821 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478559 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478561 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478564 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478567 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478572 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478577 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478580 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478583 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478587 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478590 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478593 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478596 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478598 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478601 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478604 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478625 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478629 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478632 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478634 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:54:17.480397 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478637 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478639 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478642 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478644 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478647 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478651 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478653 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478656 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478658 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478661 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478663 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478665 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478668 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478670 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478673 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478675 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478677 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478681 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478683 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478686 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:54:17.480875 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478688 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478691 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478694 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478697 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478699 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478702 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478704 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478706 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478709 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478711 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478714 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478716 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478718 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478721 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478723 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478726 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478730 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478733 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478735 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:54:17.481385 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478738 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478741 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478743 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478746 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478748 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478751 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478753 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478756 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478759 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478761 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478765 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478767 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478770 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478772 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478774 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:54:17.481933 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478777 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.478783 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478903 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478908 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478911 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478914 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478917 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478919 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478922 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478924 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478927 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478929 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478932 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478934 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478937 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:54:17.482290 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478939 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478942 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478944 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478947 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478949 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478951 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478953 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478957 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478960 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478963 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478966 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478969 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478972 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478975 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478978 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478980 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478983 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478985 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478988 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478991 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:54:17.482683 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478993 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478996 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.478998 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479000 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479003 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479007 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479010 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479013 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479015 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479018 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479021 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479025 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479028 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479030 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479033 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479035 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479038 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479040 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479043 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:54:17.483196 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479045 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479047 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479050 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479053 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479055 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479058 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479060 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479063 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479065 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479068 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479070 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479073 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479075 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479078 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479081 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479083 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479086 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479088 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479090 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:54:17.483696 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479093 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479095 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479097 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479100 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479102 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479104 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479107 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479109 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479111 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479113 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479116 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479118 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479121 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479123 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479125 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479128 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479130 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479133 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479135 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479138 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:54:17.484145 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479140 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479143 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479145 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479147 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479150 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479152 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479154 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:17.479157 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.479162 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.479274 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.482677 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.484421 2572 server.go:1019] "Starting client certificate rotation" Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.484539 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:54:17.484762 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.484584 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:54:17.487420 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.487405 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:54:17.489476 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.489453 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:54:17.498289 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.498271 2572 log.go:25] "Validated CRI v1 runtime API" Jul 14 17:54:17.500813 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.500793 2572 log.go:25] "Validated CRI v1 image API" Jul 14 17:54:17.501560 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.501544 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 17:54:17.504713 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.504694 2572 fs.go:135] Filesystem UUIDs: map[10ac928b-cadf-46e1-bb15-b0e66c8b4429:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 95f58afd-c0c2-4449-877e-4062e4d20f32:/dev/nvme0n1p3] Jul 14 17:54:17.504788 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.504712 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 17:54:17.511535 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.511417 2572 manager.go:216] Machine: {Timestamp:2026-07-14 17:54:17.510040671 +0000 UTC m=+0.372830466 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3078318 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22c1b8133f3a8995a7ef3e1da949e9 SystemUUID:ec22c1b8-133f-3a89-95a7-ef3e1da949e9 BootID:d8293deb-5ca8-42da-bebe-ba6ffd0cb28d Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a0:e1:5d:aa:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a0:e1:5d:aa:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:a4:e3:94:d8:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 17:54:17.511535 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.511527 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 17:54:17.511685 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.511673 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 17:54:17.515121 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.515094 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 17:54:17.515250 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.515123 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-115.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 17:54:17.515878 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.515868 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 17:54:17.515913 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.515879 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 17:54:17.515913 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.515887 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:54:17.515913 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.515900 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:54:17.517819 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.517804 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 17:54:17.520765 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.520741 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:54:17.521693 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521681 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:54:17.521806 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521797 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 17:54:17.521845 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521830 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 14 17:54:17.521873 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521844 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 17:54:17.521873 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521867 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 17:54:17.521922 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521881 2572 kubelet.go:398] "Adding apiserver pod source" Jul 14 17:54:17.521922 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.521892 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 17:54:17.522333 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.522322 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:54:17.522370 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.522342 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:54:17.524590 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.524574 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 17:54:17.527159 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527144 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 17:54:17.527234 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527182 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 17:54:17.527942 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527929 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527951 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527960 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527968 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527979 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527987 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.527998 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528006 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 17:54:17.528017 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528015 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 17:54:17.528253 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528024 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 17:54:17.528253 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528042 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 17:54:17.528253 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528051 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 17:54:17.528880 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528869 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 17:54:17.528939 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.528882 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 17:54:17.533412 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.533396 2572 server.go:1268] "Started kubelet" Jul 14 17:54:17.533500 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.533473 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 17:54:17.533650 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.533581 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 17:54:17.533733 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.533668 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 17:54:17.534151 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.534132 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-115.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 17:54:17.534235 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.534142 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 17:54:17.534235 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.534157 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 17:54:17.534455 ip-10-0-129-115 systemd[1]: Started Kubernetes Kubelet. Jul 14 17:54:17.535133 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.535117 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 17:54:17.536501 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.536379 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 14 17:54:17.539375 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.539346 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 17:54:17.542808 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.542790 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 17:54:17.542890 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.542805 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 17:54:17.543190 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.543147 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:17.543696 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.543683 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 17:54:17.543806 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.543798 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 17:54:17.543898 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.543683 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 17:54:17.543977 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.543964 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 14 17:54:17.543977 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.543977 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 14 17:54:17.544348 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544324 2572 factory.go:55] Registering systemd factory Jul 14 17:54:17.544348 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544341 2572 factory.go:223] Registration of the systemd container factory successfully Jul 14 17:54:17.544555 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544539 2572 factory.go:153] Registering CRI-O factory Jul 14 17:54:17.544595 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544560 2572 factory.go:223] Registration of the crio container factory successfully Jul 14 17:54:17.544682 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544670 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 17:54:17.544715 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544708 2572 factory.go:103] Registering Raw factory Jul 14 17:54:17.544741 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.544724 2572 manager.go:1196] Started watching for new ooms in manager Jul 14 17:54:17.545468 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.545449 2572 manager.go:349] Starting recovery of all containers Jul 14 17:54:17.548409 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.548368 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 17:54:17.548508 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.548448 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 17:54:17.549898 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.548475 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-115.ec2.internal.18c23987bc38827e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-115.ec2.internal,UID:ip-10-0-129-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-115.ec2.internal,},FirstTimestamp:2026-07-14 17:54:17.53332595 +0000 UTC m=+0.396115745,LastTimestamp:2026-07-14 17:54:17.53332595 +0000 UTC m=+0.396115745,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-115.ec2.internal,}" Jul 14 17:54:17.557724 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.557318 2572 manager.go:354] Recovery completed Jul 14 17:54:17.560528 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.560502 2572 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 14 17:54:17.563968 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.563956 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.568537 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.568520 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.568601 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.568551 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.568601 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.568562 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.569102 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569091 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.569163 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569091 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 17:54:17.569163 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569151 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 17:54:17.569235 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569167 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:54:17.569810 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569797 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.569852 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569822 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.569852 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.569832 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.570508 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.570498 2572 policy_none.go:49] "None policy: Start" Jul 14 17:54:17.570544 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.570513 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 17:54:17.570544 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.570525 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 17:54:17.570768 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.570706 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-115.ec2.internal.18c23987be51cfeb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-115.ec2.internal,UID:ip-10-0-129-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-115.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-115.ec2.internal,},FirstTimestamp:2026-07-14 17:54:17.568538603 +0000 UTC m=+0.431328397,LastTimestamp:2026-07-14 17:54:17.568538603 +0000 UTC m=+0.431328397,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-115.ec2.internal,}" Jul 14 17:54:17.571486 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.571474 2572 policy_none.go:47] "Start" Jul 14 17:54:17.581754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.581693 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-115.ec2.internal.18c23987be5214c6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-115.ec2.internal,UID:ip-10-0-129-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-115.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-115.ec2.internal,},FirstTimestamp:2026-07-14 17:54:17.56855623 +0000 UTC m=+0.431346026,LastTimestamp:2026-07-14 17:54:17.56855623 +0000 UTC m=+0.431346026,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-115.ec2.internal,}" Jul 14 17:54:17.590478 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.590455 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ql7tx" Jul 14 17:54:17.595924 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.595847 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-115.ec2.internal.18c23987be52385d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-115.ec2.internal,UID:ip-10-0-129-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-115.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-115.ec2.internal,},FirstTimestamp:2026-07-14 17:54:17.568565341 +0000 UTC m=+0.431355135,LastTimestamp:2026-07-14 17:54:17.568565341 +0000 UTC m=+0.431355135,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-115.ec2.internal,}" Jul 14 17:54:17.603015 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.602995 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ql7tx" Jul 14 17:54:17.612248 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612235 2572 manager.go:337] "Starting Device Plugin manager" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.612275 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612285 2572 server.go:87] "Starting device plugin registration server" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612521 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612532 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612633 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612704 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.612714 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.613245 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 17:54:17.623053 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.613305 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:17.710867 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.710791 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 17:54:17.712047 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.712028 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 17:54:17.712146 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.712071 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 17:54:17.712146 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.712094 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 17:54:17.712146 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.712129 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 17:54:17.712680 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.712662 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.713821 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.713799 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.713908 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.713834 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.713908 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.713845 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.713908 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.713868 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.715314 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.715297 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:54:17.729955 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.729939 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.730030 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.729961 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-115.ec2.internal\": node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:17.788335 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.788310 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:17.812410 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.812372 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal"] Jul 14 17:54:17.812479 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.812449 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.813431 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.813412 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.813514 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.813444 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.813514 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.813458 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.814746 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.814733 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.814850 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.814836 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.814883 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.814872 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.815498 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.815485 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.815562 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.815510 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.815562 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.815523 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.815667 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.815486 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.815667 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.815582 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.815667 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.815596 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.816673 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.816659 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.816728 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.816684 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:17.817440 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.817423 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:17.817523 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.817454 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:17.817523 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.817464 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:17.831117 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.831102 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-115.ec2.internal\" not found" node="ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.834385 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.834372 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-115.ec2.internal\" not found" node="ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.846483 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.846465 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3246bb4a5a5e8a54b4bd81e6bb6bc637-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal\" (UID: \"3246bb4a5a5e8a54b4bd81e6bb6bc637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.846548 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.846490 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3246bb4a5a5e8a54b4bd81e6bb6bc637-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal\" (UID: \"3246bb4a5a5e8a54b4bd81e6bb6bc637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.846548 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.846506 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9297309d9edb2e81185f4bef95a6103-config\") pod \"kube-apiserver-proxy-ip-10-0-129-115.ec2.internal\" (UID: \"f9297309d9edb2e81185f4bef95a6103\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.889223 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.889194 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:17.946927 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.946896 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9297309d9edb2e81185f4bef95a6103-config\") pod \"kube-apiserver-proxy-ip-10-0-129-115.ec2.internal\" (UID: \"f9297309d9edb2e81185f4bef95a6103\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.946927 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.946928 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3246bb4a5a5e8a54b4bd81e6bb6bc637-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal\" (UID: \"3246bb4a5a5e8a54b4bd81e6bb6bc637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.947087 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.946953 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3246bb4a5a5e8a54b4bd81e6bb6bc637-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal\" (UID: \"3246bb4a5a5e8a54b4bd81e6bb6bc637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.947087 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.946991 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9297309d9edb2e81185f4bef95a6103-config\") pod \"kube-apiserver-proxy-ip-10-0-129-115.ec2.internal\" (UID: \"f9297309d9edb2e81185f4bef95a6103\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.947087 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.947007 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3246bb4a5a5e8a54b4bd81e6bb6bc637-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal\" (UID: \"3246bb4a5a5e8a54b4bd81e6bb6bc637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.947087 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:17.946988 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3246bb4a5a5e8a54b4bd81e6bb6bc637-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal\" (UID: \"3246bb4a5a5e8a54b4bd81e6bb6bc637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:17.989711 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:17.989637 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.090171 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.090143 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.133350 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.133327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:18.138206 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.138190 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" Jul 14 17:54:18.190639 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.190591 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.291282 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.291198 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.391699 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.391653 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.484216 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.484177 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 17:54:18.484897 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.484343 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 17:54:18.492498 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.492466 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.570736 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.570656 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:54:18.593154 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.593131 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.607293 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.607256 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 17:49:18 +0000 UTC" deadline="2028-04-09 09:48:28.555720466 +0000 UTC" Jul 14 17:54:18.607293 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.607285 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15231h54m9.948438771s" Jul 14 17:54:18.660600 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.660577 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:54:18.693685 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.693601 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.716191 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.716163 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:54:18.717027 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.717003 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:54:18.717112 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.717043 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:54:18.717112 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.717055 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:54:18.794099 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.794065 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.794215 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:18.794140 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3246bb4a5a5e8a54b4bd81e6bb6bc637.slice/crio-5fb8885a6237c84d80d2ac7d48f06381eae174332cd32a249bd4539a9f4ab0ec WatchSource:0}: Error finding container 5fb8885a6237c84d80d2ac7d48f06381eae174332cd32a249bd4539a9f4ab0ec: Status 404 returned error can't find the container with id 5fb8885a6237c84d80d2ac7d48f06381eae174332cd32a249bd4539a9f4ab0ec Jul 14 17:54:18.800021 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:18.800008 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:54:18.895046 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.894959 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:18.995554 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:18.995529 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:19.096503 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.096487 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-115.ec2.internal\" not found" Jul 14 17:54:19.111109 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.111078 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:54:19.145888 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.145810 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" Jul 14 17:54:19.157189 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.157167 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:54:19.158146 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.158132 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" Jul 14 17:54:19.176945 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.176921 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:54:19.523214 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.523019 2572 apiserver.go:52] "Watching apiserver" Jul 14 17:54:19.534281 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.534256 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 17:54:19.534656 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.534617 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nfqj2","openshift-network-diagnostics/network-check-target-7qd6n","openshift-ovn-kubernetes/ovnkube-node-djmwx","kube-system/global-pull-secret-syncer-k2vhg","kube-system/konnectivity-agent-mxgj8","openshift-cluster-node-tuning-operator/tuned-dpcg6","openshift-dns/node-resolver-5q98w","openshift-multus/network-metrics-daemon-kmzpv","openshift-network-operator/iptables-alerter-zkcr9","kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq","openshift-image-registry/node-ca-brnmt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal","openshift-multus/multus-4wvjs"] Jul 14 17:54:19.536151 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.536128 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.537371 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.537345 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.538923 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.538896 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.540068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.540051 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.540172 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.540118 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:19.541365 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.541343 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.542657 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.542639 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.543171 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.543152 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 17:54:19.543324 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.543306 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.543460 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.543432 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b6vtx\"" Jul 14 17:54:19.543899 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.543875 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 17:54:19.544158 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.544137 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.544556 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.544392 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.545146 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.545125 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.546312 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.546290 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.546312 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.546306 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.546647 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.546629 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 17:54:19.547056 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.547036 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.547419 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.547402 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 17:54:19.547489 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.547447 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 17:54:19.547489 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.547452 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tlrs8\"" Jul 14 17:54:19.547739 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.547723 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 17:54:19.547964 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.547950 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xvdbr\"" Jul 14 17:54:19.548089 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.548068 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 17:54:19.548149 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.548065 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.548273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.548238 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.548334 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.548318 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:19.548396 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.548352 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n4vwf\"" Jul 14 17:54:19.548396 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.548365 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:19.548500 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.548483 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 17:54:19.549223 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.549197 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.551148 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.551128 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:19.551257 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.551194 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:19.552186 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.552163 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xvpwk\"" Jul 14 17:54:19.552329 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.552306 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.552399 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.552345 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tf74j\"" Jul 14 17:54:19.552457 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.552399 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.552841 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.552819 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 17:54:19.552920 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.552893 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 17:54:19.553163 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553146 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 17:54:19.553244 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553180 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.553244 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553202 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h4m9v\"" Jul 14 17:54:19.553423 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553406 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.553598 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553576 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.553716 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553588 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-kubelet\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.553716 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553661 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b715d57-a5dc-4ab9-a089-52a1c8919e91-kubelet-config\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.553716 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553696 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysctl-conf\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.553869 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/177a10f9-b388-470e-962f-d536011e1876-iptables-alerter-script\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.553869 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553769 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-sys-fs\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.553869 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553810 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-systemd-units\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.553869 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553858 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-systemd\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554036 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553893 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-var-lib-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554036 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-var-lib-kubelet\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.554036 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.553977 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-registration-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.554036 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlmns\" (UniqueName: \"kubernetes.io/projected/003566d9-20fc-4041-8563-24ee442d3130-kube-api-access-rlmns\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.554207 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-etc-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554207 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554106 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-ovn\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554509 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554148 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-log-socket\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554603 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554547 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-cni-netd\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554603 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-ovnkube-script-lib\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.554730 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554605 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0496d62-71e1-4e18-a9b2-d07898e8460e-agent-certs\") pod \"konnectivity-agent-mxgj8\" (UID: \"c0496d62-71e1-4e18-a9b2-d07898e8460e\") " pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.554730 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.554730 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-kubernetes\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.554730 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554688 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-tuned\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555095 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.554733 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbhqf\" (UniqueName: \"kubernetes.io/projected/03a7cf65-9266-4371-8921-8568aeffe9d2-kube-api-access-kbhqf\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.555165 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555165 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555148 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-node-log\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555277 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-cni-bin\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555277 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555215 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-modprobe-d\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555277 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kf2k\" (UniqueName: \"kubernetes.io/projected/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-kube-api-access-9kf2k\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555277 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555263 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-socket-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-etc-selinux\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555320 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-slash\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555371 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-env-overrides\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555421 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42a85528-7705-48be-95f0-5e7275d8f9d3-ovn-node-metrics-cert\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0496d62-71e1-4e18-a9b2-d07898e8460e-konnectivity-ca\") pod \"konnectivity-agent-mxgj8\" (UID: \"c0496d62-71e1-4e18-a9b2-d07898e8460e\") " pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555452 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysconfig\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555466 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-systemd\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555477 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-lib-modules\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/177a10f9-b388-470e-962f-d536011e1876-host-slash\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-run-netns\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555581 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-ovnkube-config\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555605 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-tmp\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555650 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03a7cf65-9266-4371-8921-8568aeffe9d2-hosts-file\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555675 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-device-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555702 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kjzl\" (UniqueName: \"kubernetes.io/projected/42a85528-7705-48be-95f0-5e7275d8f9d3-kube-api-access-4kjzl\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555753 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysctl-d\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555777 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a7cf65-9266-4371-8921-8568aeffe9d2-tmp-dir\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555800 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b715d57-a5dc-4ab9-a089-52a1c8919e91-dbus\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555871 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555904 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-run\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.555938 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555930 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-sys\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.556349 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555953 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-host\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.556349 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.555982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mcrs\" (UniqueName: \"kubernetes.io/projected/177a10f9-b388-470e-962f-d536011e1876-kube-api-access-4mcrs\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.556349 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.556005 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.559483 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.559459 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 17:54:19.559583 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.559493 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 17:54:19.559665 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.559623 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-297lb\"" Jul 14 17:54:19.560420 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.560390 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 17:54:19.560420 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.560391 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 17:54:19.562194 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.561791 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 17:54:19.562194 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.562036 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9tts\"" Jul 14 17:54:19.566148 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.566128 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:54:19.606461 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.606434 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qngsd" Jul 14 17:54:19.620907 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.620880 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qngsd" Jul 14 17:54:19.645676 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.645649 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 17:54:19.657155 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657126 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0496d62-71e1-4e18-a9b2-d07898e8460e-agent-certs\") pod \"konnectivity-agent-mxgj8\" (UID: \"c0496d62-71e1-4e18-a9b2-d07898e8460e\") " pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.657296 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-kubernetes\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.657296 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657256 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-cni-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.657296 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657263 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-kubernetes\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.657450 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657281 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-netns\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.657450 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657347 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b55e97e6-bdd1-435b-8a18-1975c8640be7-serviceca\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.657450 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657373 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-modprobe-d\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.657450 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-socket-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-env-overrides\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657521 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-os-release\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657547 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657563 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-modprobe-d\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657555 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0496d62-71e1-4e18-a9b2-d07898e8460e-konnectivity-ca\") pod \"konnectivity-agent-mxgj8\" (UID: \"c0496d62-71e1-4e18-a9b2-d07898e8460e\") " pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.657669 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-lib-modules\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-run-netns\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657707 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-socket-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657721 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-cni-binary-copy\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657755 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-run-netns\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657773 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03a7cf65-9266-4371-8921-8568aeffe9d2-hosts-file\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657816 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-device-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657850 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-lib-modules\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657851 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03a7cf65-9266-4371-8921-8568aeffe9d2-hosts-file\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kjzl\" (UniqueName: \"kubernetes.io/projected/42a85528-7705-48be-95f0-5e7275d8f9d3-kube-api-access-4kjzl\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657909 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-device-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.658068 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.657940 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-kubelet\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658080 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658090 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-env-overrides\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658116 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658203 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysctl-d\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658231 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a7cf65-9266-4371-8921-8568aeffe9d2-tmp-dir\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658217 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0496d62-71e1-4e18-a9b2-d07898e8460e-konnectivity-ca\") pod \"konnectivity-agent-mxgj8\" (UID: \"c0496d62-71e1-4e18-a9b2-d07898e8460e\") " pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-cnibin\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658286 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-k8s-cni-cncf-io\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658312 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-system-cni-dir\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658318 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysctl-d\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658339 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-run\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658369 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-host\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658389 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-run\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mcrs\" (UniqueName: \"kubernetes.io/projected/177a10f9-b388-470e-962f-d536011e1876-kube-api-access-4mcrs\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658439 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-host\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658444 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.658633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658472 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-kubelet\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-systemd\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658505 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658523 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a7cf65-9266-4371-8921-8568aeffe9d2-tmp-dir\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658559 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-var-lib-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658561 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-kubelet\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658601 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-systemd\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-var-lib-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b715d57-a5dc-4ab9-a089-52a1c8919e91-kubelet-config\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-conf-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658784 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-cnibin\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658828 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qfp\" (UniqueName: \"kubernetes.io/projected/c264af97-795e-43eb-84e3-81aeab4aacf6-kube-api-access-q5qfp\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b715d57-a5dc-4ab9-a089-52a1c8919e91-kubelet-config\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658871 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-var-lib-kubelet\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658904 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-registration-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658928 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlmns\" (UniqueName: \"kubernetes.io/projected/003566d9-20fc-4041-8563-24ee442d3130-kube-api-access-rlmns\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658948 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-var-lib-kubelet\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.659495 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658978 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-registration-dir\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.658983 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-ovn\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659018 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-ovn\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659023 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-hostroot\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659052 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-multus-certs\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659090 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gcr\" (UniqueName: \"kubernetes.io/projected/b55e97e6-bdd1-435b-8a18-1975c8640be7-kube-api-access-v9gcr\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-tuned\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659124 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbhqf\" (UniqueName: \"kubernetes.io/projected/03a7cf65-9266-4371-8921-8568aeffe9d2-kube-api-access-kbhqf\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42a85528-7705-48be-95f0-5e7275d8f9d3-ovn-node-metrics-cert\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659173 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-etc-kubernetes\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659200 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b55e97e6-bdd1-435b-8a18-1975c8640be7-host\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659216 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-system-cni-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659247 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kf2k\" (UniqueName: \"kubernetes.io/projected/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-kube-api-access-9kf2k\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659273 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-etc-selinux\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659314 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-slash\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659331 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.660499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btkrb\" (UniqueName: \"kubernetes.io/projected/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-kube-api-access-btkrb\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659365 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysconfig\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-systemd\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659402 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/177a10f9-b388-470e-962f-d536011e1876-host-slash\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659426 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-ovnkube-config\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-tmp\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659472 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659505 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-cni-bin\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659545 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-daemon-config\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659575 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-socket-dir-parent\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659602 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b715d57-a5dc-4ab9-a089-52a1c8919e91-dbus\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659626 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-sys\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-systemd-units\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659716 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659741 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-node-log\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659765 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-cni-bin\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.661407 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659819 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.659995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysctl-conf\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/177a10f9-b388-470e-962f-d536011e1876-iptables-alerter-script\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660052 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-sys-fs\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-etc-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-cni-netd\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660132 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-cni-multus\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660159 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkqfv\" (UniqueName: \"kubernetes.io/projected/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-kube-api-access-jkqfv\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660184 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-log-socket\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-ovnkube-script-lib\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660237 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-os-release\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660567 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-etc-selinux\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660622 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-slash\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660669 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660758 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysconfig\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660813 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-systemd\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.660856 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/177a10f9-b388-470e-962f-d536011e1876-host-slash\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.662246 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661366 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-ovnkube-config\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661558 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0496d62-71e1-4e18-a9b2-d07898e8460e-agent-certs\") pod \"konnectivity-agent-mxgj8\" (UID: \"c0496d62-71e1-4e18-a9b2-d07898e8460e\") " pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661645 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-cni-bin\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661724 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-etc-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661791 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-cni-netd\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661798 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661833 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b715d57-a5dc-4ab9-a089-52a1c8919e91-dbus\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661884 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-systemd-units\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661918 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-sys\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.661940 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-run-openvswitch\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.661995 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662000 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-sysctl-conf\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662030 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/003566d9-20fc-4041-8563-24ee442d3130-sys-fs\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.662057 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:20.162036317 +0000 UTC m=+3.024826107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662082 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-log-socket\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662128 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42a85528-7705-48be-95f0-5e7275d8f9d3-node-log\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662216 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-etc-tuned\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.662961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662449 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/177a10f9-b388-470e-962f-d536011e1876-iptables-alerter-script\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.663611 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662603 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42a85528-7705-48be-95f0-5e7275d8f9d3-ovnkube-script-lib\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.663611 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.662903 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42a85528-7705-48be-95f0-5e7275d8f9d3-ovn-node-metrics-cert\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.663706 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.663651 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-tmp\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.667381 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.667354 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mcrs\" (UniqueName: \"kubernetes.io/projected/177a10f9-b388-470e-962f-d536011e1876-kube-api-access-4mcrs\") pod \"iptables-alerter-zkcr9\" (UID: \"177a10f9-b388-470e-962f-d536011e1876\") " pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.668224 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.668202 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kjzl\" (UniqueName: \"kubernetes.io/projected/42a85528-7705-48be-95f0-5e7275d8f9d3-kube-api-access-4kjzl\") pod \"ovnkube-node-djmwx\" (UID: \"42a85528-7705-48be-95f0-5e7275d8f9d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.669390 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.669373 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:54:19.671348 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.671328 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbhqf\" (UniqueName: \"kubernetes.io/projected/03a7cf65-9266-4371-8921-8568aeffe9d2-kube-api-access-kbhqf\") pod \"node-resolver-5q98w\" (UID: \"03a7cf65-9266-4371-8921-8568aeffe9d2\") " pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.673886 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.673865 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlmns\" (UniqueName: \"kubernetes.io/projected/003566d9-20fc-4041-8563-24ee442d3130-kube-api-access-rlmns\") pod \"aws-ebs-csi-driver-node-r7wsq\" (UID: \"003566d9-20fc-4041-8563-24ee442d3130\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.674486 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.674464 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kf2k\" (UniqueName: \"kubernetes.io/projected/b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c-kube-api-access-9kf2k\") pod \"tuned-dpcg6\" (UID: \"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c\") " pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.717531 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.717473 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" event={"ID":"3246bb4a5a5e8a54b4bd81e6bb6bc637","Type":"ContainerStarted","Data":"5fb8885a6237c84d80d2ac7d48f06381eae174332cd32a249bd4539a9f4ab0ec"} Jul 14 17:54:19.761591 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:19.761591 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761596 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-cni-multus\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkqfv\" (UniqueName: \"kubernetes.io/projected/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-kube-api-access-jkqfv\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761657 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-os-release\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-cni-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-netns\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761731 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b55e97e6-bdd1-435b-8a18-1975c8640be7-serviceca\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761756 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761782 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-os-release\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761805 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761831 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-cni-binary-copy\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-kubelet\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761913 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-netns\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.761931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761934 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-cnibin\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761958 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-k8s-cni-cncf-io\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-cni-multus\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.761985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-system-cni-dir\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762018 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-conf-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-cnibin\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qfp\" (UniqueName: \"kubernetes.io/projected/c264af97-795e-43eb-84e3-81aeab4aacf6-kube-api-access-q5qfp\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762146 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-hostroot\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762159 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-kubelet\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762208 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-os-release\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762239 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-multus-certs\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762271 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-cni-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762443 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.762525 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:19.762646 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.762584 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:20.262565036 +0000 UTC m=+3.125354837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762715 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b55e97e6-bdd1-435b-8a18-1975c8640be7-serviceca\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762837 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-os-release\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762856 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762920 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-cnibin\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762962 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-k8s-cni-cncf-io\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763005 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-system-cni-dir\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763050 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-conf-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763114 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c264af97-795e-43eb-84e3-81aeab4aacf6-cnibin\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763118 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.762168 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-run-multus-certs\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763182 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gcr\" (UniqueName: \"kubernetes.io/projected/b55e97e6-bdd1-435b-8a18-1975c8640be7-kube-api-access-v9gcr\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763205 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-etc-kubernetes\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763220 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b55e97e6-bdd1-435b-8a18-1975c8640be7-host\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763236 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763256 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-system-cni-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btkrb\" (UniqueName: \"kubernetes.io/projected/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-kube-api-access-btkrb\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:19.763362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763321 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-cni-bin\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763349 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-daemon-config\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763351 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-hostroot\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763369 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-cni-binary-copy\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-socket-dir-parent\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763439 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-host-var-lib-cni-bin\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763482 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-socket-dir-parent\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763716 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-system-cni-dir\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763766 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-etc-kubernetes\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763841 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-multus-daemon-config\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763842 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c264af97-795e-43eb-84e3-81aeab4aacf6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.764273 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.763881 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b55e97e6-bdd1-435b-8a18-1975c8640be7-host\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.776715 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.776636 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:19.776715 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.776671 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:19.776715 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.776684 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:19.776965 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:19.776783 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:20.276768217 +0000 UTC m=+3.139558021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:19.782453 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.779737 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkqfv\" (UniqueName: \"kubernetes.io/projected/ab5bd4d6-3e97-4f12-95f3-4af553b230a9-kube-api-access-jkqfv\") pod \"multus-4wvjs\" (UID: \"ab5bd4d6-3e97-4f12-95f3-4af553b230a9\") " pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.782453 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.780136 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gcr\" (UniqueName: \"kubernetes.io/projected/b55e97e6-bdd1-435b-8a18-1975c8640be7-kube-api-access-v9gcr\") pod \"node-ca-brnmt\" (UID: \"b55e97e6-bdd1-435b-8a18-1975c8640be7\") " pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:19.782453 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.780136 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qfp\" (UniqueName: \"kubernetes.io/projected/c264af97-795e-43eb-84e3-81aeab4aacf6-kube-api-access-q5qfp\") pod \"multus-additional-cni-plugins-nfqj2\" (UID: \"c264af97-795e-43eb-84e3-81aeab4aacf6\") " pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.782453 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.781797 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btkrb\" (UniqueName: \"kubernetes.io/projected/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-kube-api-access-btkrb\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:19.852935 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.852897 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5q98w" Jul 14 17:54:19.855543 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.855516 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" Jul 14 17:54:19.868001 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.867976 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4wvjs" Jul 14 17:54:19.873133 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.873111 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:19.879983 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.879960 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:19.886724 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.886702 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" Jul 14 17:54:19.894709 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.894685 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zkcr9" Jul 14 17:54:19.901319 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.901302 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" Jul 14 17:54:19.907095 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:19.907070 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-brnmt" Jul 14 17:54:20.045890 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.045720 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a7cf65_9266_4371_8921_8568aeffe9d2.slice/crio-6e066ac172db084f20f6a77a4cdaf5183f6cbf490f41d19f430c7354a8376ec1 WatchSource:0}: Error finding container 6e066ac172db084f20f6a77a4cdaf5183f6cbf490f41d19f430c7354a8376ec1: Status 404 returned error can't find the container with id 6e066ac172db084f20f6a77a4cdaf5183f6cbf490f41d19f430c7354a8376ec1 Jul 14 17:54:20.049108 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.049078 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09f6f61_8a67_4a93_9bd5_cfc7ded17b2c.slice/crio-0173b42dbdab8340b311fd70cac45cb6a5b9b1f67258fe857ffe51c54e12e737 WatchSource:0}: Error finding container 0173b42dbdab8340b311fd70cac45cb6a5b9b1f67258fe857ffe51c54e12e737: Status 404 returned error can't find the container with id 0173b42dbdab8340b311fd70cac45cb6a5b9b1f67258fe857ffe51c54e12e737 Jul 14 17:54:20.050214 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.050191 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc264af97_795e_43eb_84e3_81aeab4aacf6.slice/crio-1a306ed9b321a9ee5a4dca2b6e52d04cef2f7afba95601694b8a4ea722763816 WatchSource:0}: Error finding container 1a306ed9b321a9ee5a4dca2b6e52d04cef2f7afba95601694b8a4ea722763816: Status 404 returned error can't find the container with id 1a306ed9b321a9ee5a4dca2b6e52d04cef2f7afba95601694b8a4ea722763816 Jul 14 17:54:20.051020 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.050990 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod003566d9_20fc_4041_8563_24ee442d3130.slice/crio-930eaa43c53f13a9eda138ecc1cc792578d990399341931c219da125298b4b6f WatchSource:0}: Error finding container 930eaa43c53f13a9eda138ecc1cc792578d990399341931c219da125298b4b6f: Status 404 returned error can't find the container with id 930eaa43c53f13a9eda138ecc1cc792578d990399341931c219da125298b4b6f Jul 14 17:54:20.051985 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.051952 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177a10f9_b388_470e_962f_d536011e1876.slice/crio-29a8643667e89019cf2c376fd92bf611eaa666474dc147e0f64df3a423984d13 WatchSource:0}: Error finding container 29a8643667e89019cf2c376fd92bf611eaa666474dc147e0f64df3a423984d13: Status 404 returned error can't find the container with id 29a8643667e89019cf2c376fd92bf611eaa666474dc147e0f64df3a423984d13 Jul 14 17:54:20.052832 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.052809 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55e97e6_bdd1_435b_8a18_1975c8640be7.slice/crio-eed4ac8cefbc28ea92a81b674ae574de7b4924fff41d1de401c487e59a1681f6 WatchSource:0}: Error finding container eed4ac8cefbc28ea92a81b674ae574de7b4924fff41d1de401c487e59a1681f6: Status 404 returned error can't find the container with id eed4ac8cefbc28ea92a81b674ae574de7b4924fff41d1de401c487e59a1681f6 Jul 14 17:54:20.055398 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.055022 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5bd4d6_3e97_4f12_95f3_4af553b230a9.slice/crio-f2a6ddaec3d777163415bf1315bd2c3acc9b07230c5e8ed7139cd09073936780 WatchSource:0}: Error finding container f2a6ddaec3d777163415bf1315bd2c3acc9b07230c5e8ed7139cd09073936780: Status 404 returned error can't find the container with id f2a6ddaec3d777163415bf1315bd2c3acc9b07230c5e8ed7139cd09073936780 Jul 14 17:54:20.055733 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.055667 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a85528_7705_48be_95f0_5e7275d8f9d3.slice/crio-a523f0c69b6aa90d9f227aed6672077af1f9df8637b58d0949e896f305044d9d WatchSource:0}: Error finding container a523f0c69b6aa90d9f227aed6672077af1f9df8637b58d0949e896f305044d9d: Status 404 returned error can't find the container with id a523f0c69b6aa90d9f227aed6672077af1f9df8637b58d0949e896f305044d9d Jul 14 17:54:20.056688 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:20.056604 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0496d62_71e1_4e18_a9b2_d07898e8460e.slice/crio-a18e3ccfbc29ac95cf5078535590bf74e05b793c383263f224b05026b48c0f65 WatchSource:0}: Error finding container a18e3ccfbc29ac95cf5078535590bf74e05b793c383263f224b05026b48c0f65: Status 404 returned error can't find the container with id a18e3ccfbc29ac95cf5078535590bf74e05b793c383263f224b05026b48c0f65 Jul 14 17:54:20.167016 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.166984 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:20.167174 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.167157 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:20.167237 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.167226 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:21.167209627 +0000 UTC m=+4.029999428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:20.267659 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.267611 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:20.267784 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.267763 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:20.267831 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.267821 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:21.267805507 +0000 UTC m=+4.130595313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:20.368439 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.368407 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:20.368553 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.368527 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:20.368553 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.368541 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:20.368553 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.368553 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:20.368699 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.368603 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:21.368589571 +0000 UTC m=+4.231379368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:20.623486 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.623386 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:49:20 +0000 UTC" deadline="2028-04-28 16:27:36.222390954 +0000 UTC" Jul 14 17:54:20.623486 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.623425 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15694h33m15.598971424s" Jul 14 17:54:20.712964 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.712931 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:20.713132 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:20.713073 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:20.740840 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.739850 2572 generic.go:356] "Generic (PLEG): container finished" podID="3246bb4a5a5e8a54b4bd81e6bb6bc637" containerID="53f9a031e36e551f7e148a0a8771fe2b30719c64b8066e10c8f57c169f7871db" exitCode=0 Jul 14 17:54:20.740840 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.739936 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" event={"ID":"3246bb4a5a5e8a54b4bd81e6bb6bc637","Type":"ContainerDied","Data":"53f9a031e36e551f7e148a0a8771fe2b30719c64b8066e10c8f57c169f7871db"} Jul 14 17:54:20.761844 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.761814 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mxgj8" event={"ID":"c0496d62-71e1-4e18-a9b2-d07898e8460e","Type":"ContainerStarted","Data":"a18e3ccfbc29ac95cf5078535590bf74e05b793c383263f224b05026b48c0f65"} Jul 14 17:54:20.773890 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.773860 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"a523f0c69b6aa90d9f227aed6672077af1f9df8637b58d0949e896f305044d9d"} Jul 14 17:54:20.785182 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.785152 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4wvjs" event={"ID":"ab5bd4d6-3e97-4f12-95f3-4af553b230a9","Type":"ContainerStarted","Data":"f2a6ddaec3d777163415bf1315bd2c3acc9b07230c5e8ed7139cd09073936780"} Jul 14 17:54:20.795890 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.795859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zkcr9" event={"ID":"177a10f9-b388-470e-962f-d536011e1876","Type":"ContainerStarted","Data":"29a8643667e89019cf2c376fd92bf611eaa666474dc147e0f64df3a423984d13"} Jul 14 17:54:20.800560 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.800453 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerStarted","Data":"1a306ed9b321a9ee5a4dca2b6e52d04cef2f7afba95601694b8a4ea722763816"} Jul 14 17:54:20.804064 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.803993 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" event={"ID":"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c","Type":"ContainerStarted","Data":"0173b42dbdab8340b311fd70cac45cb6a5b9b1f67258fe857ffe51c54e12e737"} Jul 14 17:54:20.807125 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.807087 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" event={"ID":"f9297309d9edb2e81185f4bef95a6103","Type":"ContainerStarted","Data":"df4fc6791706d0e330457a2c298cf5bfb818155c8f656f2bd5a3b57afc9b81da"} Jul 14 17:54:20.808935 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.808908 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-brnmt" event={"ID":"b55e97e6-bdd1-435b-8a18-1975c8640be7","Type":"ContainerStarted","Data":"eed4ac8cefbc28ea92a81b674ae574de7b4924fff41d1de401c487e59a1681f6"} Jul 14 17:54:20.811930 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.811902 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" event={"ID":"003566d9-20fc-4041-8563-24ee442d3130","Type":"ContainerStarted","Data":"930eaa43c53f13a9eda138ecc1cc792578d990399341931c219da125298b4b6f"} Jul 14 17:54:20.829831 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:20.829751 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5q98w" event={"ID":"03a7cf65-9266-4371-8921-8568aeffe9d2","Type":"ContainerStarted","Data":"6e066ac172db084f20f6a77a4cdaf5183f6cbf490f41d19f430c7354a8376ec1"} Jul 14 17:54:21.175515 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.175421 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:21.175739 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.175568 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:21.175739 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.175634 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:23.175614088 +0000 UTC m=+6.038403884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:21.276881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.276840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:21.277112 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.277071 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:21.277182 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.277144 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:23.277121944 +0000 UTC m=+6.139911743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:21.377914 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.377878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:21.378100 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.378084 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:21.378166 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.378112 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:21.378166 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.378126 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:21.378245 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.378187 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:23.378168466 +0000 UTC m=+6.240958253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:21.624558 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.624412 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:49:20 +0000 UTC" deadline="2028-04-27 15:19:56.159209516 +0000 UTC" Jul 14 17:54:21.624558 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.624478 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15669h25m34.534737236s" Jul 14 17:54:21.714049 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.713279 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:21.714049 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.713432 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:21.714049 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.713918 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:21.714049 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:21.714009 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:21.851020 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:21.850249 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" event={"ID":"3246bb4a5a5e8a54b4bd81e6bb6bc637","Type":"ContainerStarted","Data":"448b61ad92b41d12f5978e32178a4911924b45550ef9e0212a135387b8ca524b"} Jul 14 17:54:22.713580 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:22.713037 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:22.713580 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:22.713175 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:23.195180 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:23.195079 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:23.195361 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.195252 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:23.195361 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.195315 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:27.195296936 +0000 UTC m=+10.058086719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:23.296303 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:23.296262 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:23.296486 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.296448 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:23.296565 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.296514 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:27.296493974 +0000 UTC m=+10.159283758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:23.397281 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:23.397238 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:23.397520 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.397484 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:23.397520 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.397513 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:23.397695 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.397527 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:23.397695 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.397586 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:27.397567657 +0000 UTC m=+10.260357445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:23.714310 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:23.713773 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:23.714310 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.713908 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:23.715107 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:23.714948 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:23.715107 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:23.715069 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:24.713214 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:24.712669 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:24.713214 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:24.712811 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:25.715414 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:25.715377 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:25.715865 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:25.715388 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:25.715865 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:25.715512 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:25.715865 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:25.715594 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:26.713242 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:26.713215 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:26.713417 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:26.713332 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:27.230870 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:27.230275 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:27.230870 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.230468 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:27.230870 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.230528 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:35.230510681 +0000 UTC m=+18.093300466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:27.331406 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:27.331374 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:27.331584 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.331494 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:27.331584 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.331555 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:35.331538154 +0000 UTC m=+18.194327952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:27.432561 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:27.431983 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:27.432561 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.432145 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:27.432561 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.432167 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:27.432561 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.432180 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:27.432561 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.432232 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:35.43221488 +0000 UTC m=+18.295004666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:27.714599 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:27.713301 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:27.714599 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.713434 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:27.714599 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:27.713863 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:27.714599 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:27.713968 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:28.713227 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:28.713182 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:28.713646 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:28.713320 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:29.714416 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:29.712513 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:29.714416 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:29.712566 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:29.714416 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:29.712685 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:29.714416 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:29.712847 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:30.712281 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:30.712246 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:30.712447 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:30.712370 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:31.713163 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:31.713134 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:31.713645 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:31.713243 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:31.713645 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:31.713300 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:31.713645 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:31.713403 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:32.712527 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:32.712498 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:32.712692 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:32.712632 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:33.713352 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:33.713316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:33.713868 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:33.713316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:33.713868 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:33.713435 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:33.713868 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:33.713522 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:34.712596 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:34.712553 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:34.712851 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:34.712695 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:35.302666 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:35.302624 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:35.303170 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.302777 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:35.303170 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.302856 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:51.302836814 +0000 UTC m=+34.165626609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:35.403712 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:35.403671 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:35.403879 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.403815 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:35.403879 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.403875 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:51.403857417 +0000 UTC m=+34.266647207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:35.505111 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:35.505068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:35.505283 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.505226 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:35.505283 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.505245 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:35.505283 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.505255 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:35.505445 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.505315 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:51.505296361 +0000 UTC m=+34.368086164 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:35.713376 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:35.713299 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:35.713528 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.713412 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:35.713528 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:35.713482 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:35.713660 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:35.713571 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:36.712719 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:36.712682 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:36.713139 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:36.713102 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:37.713743 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:37.713710 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:37.714184 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:37.713815 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:37.714184 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:37.714140 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:37.714306 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:37.714256 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:38.712506 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:38.712473 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:38.712714 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:38.712584 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:39.712704 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:39.712666 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:39.713063 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:39.712810 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:39.713321 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:39.713295 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:39.713437 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:39.713414 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:40.712406 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.712241 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:40.712503 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:40.712483 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:40.902805 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.902727 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4wvjs" event={"ID":"ab5bd4d6-3e97-4f12-95f3-4af553b230a9","Type":"ContainerStarted","Data":"29a64934b6c1eaaf8954f1c8c2ca23e38ce75194346124a65f9050d9b008053a"} Jul 14 17:54:40.904264 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.904243 2572 generic.go:356] "Generic (PLEG): container finished" podID="c264af97-795e-43eb-84e3-81aeab4aacf6" containerID="b7f36dd9063468d9e22c3bd8dfbee590376c0791b1f4874c40e6cd68cff43d73" exitCode=0 Jul 14 17:54:40.904357 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.904308 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerDied","Data":"b7f36dd9063468d9e22c3bd8dfbee590376c0791b1f4874c40e6cd68cff43d73"} Jul 14 17:54:40.906268 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.906244 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" event={"ID":"b09f6f61-8a67-4a93-9bd5-cfc7ded17b2c","Type":"ContainerStarted","Data":"718c38a9e70f17828be0c2a00e3f6f4e577891fb329052c6da21f1a39c32e4be"} Jul 14 17:54:40.907935 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.907913 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" event={"ID":"f9297309d9edb2e81185f4bef95a6103","Type":"ContainerStarted","Data":"dee37f6349ac324ae22c90326430bf588fbfb1e02dd8374b1f10b981d871eaa7"} Jul 14 17:54:40.909333 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.909313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-brnmt" event={"ID":"b55e97e6-bdd1-435b-8a18-1975c8640be7","Type":"ContainerStarted","Data":"019992a4a0d37cd36b65e759b5df4463bf9dc092dad30bb1ca3ed648f523a262"} Jul 14 17:54:40.910735 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.910705 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" event={"ID":"003566d9-20fc-4041-8563-24ee442d3130","Type":"ContainerStarted","Data":"3d9bf2fab2f566e7b95a75cfd53a21c954a73acf55b416ea2f5171b8b542bb8e"} Jul 14 17:54:40.912089 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.912069 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5q98w" event={"ID":"03a7cf65-9266-4371-8921-8568aeffe9d2","Type":"ContainerStarted","Data":"e2f74467e4ec2282af2992fa25efc87b72a008a5c3e4935b0c62dbb4de12bb74"} Jul 14 17:54:40.913294 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.913276 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mxgj8" event={"ID":"c0496d62-71e1-4e18-a9b2-d07898e8460e","Type":"ContainerStarted","Data":"3dc818defda8dd210ca84d223f7163766fbdee16452cd3d64fe3d7d9db25553d"} Jul 14 17:54:40.916195 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.916174 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"aef4cee477df6da2a7e8d2e9e8edfdcbd35868440a4a6284b1798e52179c3859"} Jul 14 17:54:40.916274 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.916251 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"e856f64ab98eed3856801757dee9407153fbcc2cb308c535d0b1585a124e92a1"} Jul 14 17:54:40.916313 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.916302 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"eba0a2fbc844cf135b1fcec7fafa45866f6a5b445d0fb8dfd0773b8a8af3eec8"} Jul 14 17:54:40.916362 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.916350 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"308c540dd7dca91ba962d19bf5914e384901076fdf249751f0b35faf8f0f9b61"} Jul 14 17:54:40.916462 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.916446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"cfbec612a0bc5ef139d401622edb5b7e58e3b3d3d79e98780a38102aad30e7e8"} Jul 14 17:54:40.916516 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.916504 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"1f8e114bd47c132048dc7cc2d5b9e02e3e36b85596d1f858d689111c6e59d616"} Jul 14 17:54:40.932861 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.932821 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-115.ec2.internal" podStartSLOduration=21.925546119 podStartE2EDuration="21.925546119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:21.872794823 +0000 UTC m=+4.735584644" watchObservedRunningTime="2026-07-14 17:54:40.925546119 +0000 UTC m=+23.788335924" Jul 14 17:54:40.932943 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.932894 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4wvjs" podStartSLOduration=2.808204058 podStartE2EDuration="22.932887114s" totalImagesPullingTime="20.124683056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:40.925324244 +0000 UTC m=+23.788114049" watchObservedRunningTime="2026-07-14 17:54:40.932887114 +0000 UTC m=+23.795676918" Jul 14 17:54:40.943011 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.942971 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-brnmt" podStartSLOduration=2.8715669630000002 podStartE2EDuration="22.942959154s" totalImagesPullingTime="20.071392191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:40.942370466 +0000 UTC m=+23.805160267" watchObservedRunningTime="2026-07-14 17:54:40.942959154 +0000 UTC m=+23.805748957" Jul 14 17:54:40.958988 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.958955 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-115.ec2.internal" podStartSLOduration=21.958944441 podStartE2EDuration="21.958944441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:40.958516656 +0000 UTC m=+23.821306459" watchObservedRunningTime="2026-07-14 17:54:40.958944441 +0000 UTC m=+23.821734244" Jul 14 17:54:40.976186 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.976136 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5q98w" podStartSLOduration=2.8963453230000002 podStartE2EDuration="22.97612316s" totalImagesPullingTime="20.079777837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:40.97538949 +0000 UTC m=+23.838179294" watchObservedRunningTime="2026-07-14 17:54:40.97612316 +0000 UTC m=+23.838912969" Jul 14 17:54:40.996205 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:40.996159 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mxgj8" podStartSLOduration=11.097322342 podStartE2EDuration="22.996145287s" totalImagesPullingTime="11.898822945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:40.99569973 +0000 UTC m=+23.858489534" watchObservedRunningTime="2026-07-14 17:54:40.996145287 +0000 UTC m=+23.858935092" Jul 14 17:54:41.015660 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:41.015583 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dpcg6" podStartSLOduration=2.937265168 podStartE2EDuration="23.015573118s" totalImagesPullingTime="20.07830795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:41.015393355 +0000 UTC m=+23.878183162" watchObservedRunningTime="2026-07-14 17:54:41.015573118 +0000 UTC m=+23.878362922" Jul 14 17:54:41.714947 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:41.714917 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:41.715100 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:41.715014 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:41.715374 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:41.715351 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:41.715498 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:41.715465 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:41.786406 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:41.786382 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 17:54:41.920207 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:41.919991 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zkcr9" event={"ID":"177a10f9-b388-470e-962f-d536011e1876","Type":"ContainerStarted","Data":"66103d9c79692d40b6c15ad810983f4370fd5f41952a8722bcfefb12bfcaceed"} Jul 14 17:54:41.921784 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:41.921760 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" event={"ID":"003566d9-20fc-4041-8563-24ee442d3130","Type":"ContainerStarted","Data":"d46b2e2405d07fc99ea1ee44c953f4031fc634ce69358e850ac2ef4cde90870d"} Jul 14 17:54:42.623441 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.623353 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:54:41.786401956Z","UUID":"2e79bdf9-693d-4b51-a8f0-62f7bd15e735","Handler":null,"Name":"","Endpoint":""} Jul 14 17:54:42.624817 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.624796 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 17:54:42.624931 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.624836 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 17:54:42.712717 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.712686 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:42.712840 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:42.712808 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:42.925759 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.925727 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" event={"ID":"003566d9-20fc-4041-8563-24ee442d3130","Type":"ContainerStarted","Data":"77460f83541898a3288e425af62766a83e2a60c879cd1faa5fdb8a27e9b55782"} Jul 14 17:54:42.947228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.947186 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zkcr9" podStartSLOduration=4.881022721 podStartE2EDuration="24.947172323s" totalImagesPullingTime="20.066149602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:41.953447791 +0000 UTC m=+24.816237607" watchObservedRunningTime="2026-07-14 17:54:42.947172323 +0000 UTC m=+25.809962127" Jul 14 17:54:42.947312 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:42.947273 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r7wsq" podStartSLOduration=2.651254894 podStartE2EDuration="24.947269877s" totalImagesPullingTime="22.296014983s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:42.946973114 +0000 UTC m=+25.809762919" watchObservedRunningTime="2026-07-14 17:54:42.947269877 +0000 UTC m=+25.810059681" Jul 14 17:54:43.712691 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:43.712655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:43.712966 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:43.712659 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:43.712966 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:43.712760 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:43.712966 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:43.712857 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:43.763207 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:43.763112 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:43.931905 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:43.931865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"bd9b31a613b0381cea3f6905e2b71c0d1f6f818247af3ba6530008217ea4f530"} Jul 14 17:54:44.713204 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:44.713174 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:44.713387 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:44.713294 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:44.971160 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:44.971078 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:44.971864 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:44.971846 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:45.716442 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:45.716197 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:45.716656 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:45.716243 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:45.716656 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:45.716560 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:45.716759 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:45.716650 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:45.941542 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:45.941057 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" event={"ID":"42a85528-7705-48be-95f0-5e7275d8f9d3","Type":"ContainerStarted","Data":"cba71706bb5727e599b9af2e36402ac08f92ea8686cda5a7669dce85b0d83957"} Jul 14 17:54:45.941799 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:45.941781 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mxgj8" Jul 14 17:54:45.980855 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:45.980754 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" podStartSLOduration=7.846800835 podStartE2EDuration="27.980735242s" totalImagesPullingTime="20.133934407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:45.979549636 +0000 UTC m=+28.842339442" watchObservedRunningTime="2026-07-14 17:54:45.980735242 +0000 UTC m=+28.843525046" Jul 14 17:54:46.713186 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:46.713151 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:46.713375 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:46.713291 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:46.944341 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:46.944304 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:46.944499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:46.944388 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:46.944499 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:46.944441 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:46.962918 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:46.962889 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:46.963135 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:46.963116 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:54:47.354741 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.354714 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k2vhg"] Jul 14 17:54:47.355087 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.354811 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:47.355087 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:47.354910 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:47.356814 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.356763 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7qd6n"] Jul 14 17:54:47.356936 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.356877 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:47.357005 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:47.356958 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:47.357378 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.357350 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kmzpv"] Jul 14 17:54:47.357484 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.357459 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:47.357587 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:47.357568 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:47.947505 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.947469 2572 generic.go:356] "Generic (PLEG): container finished" podID="c264af97-795e-43eb-84e3-81aeab4aacf6" containerID="895c5a3b8dab69974e8dbc35290d8491678a8a7e39b92ad5e3a7d353538867fd" exitCode=0 Jul 14 17:54:47.947681 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:47.947562 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerDied","Data":"895c5a3b8dab69974e8dbc35290d8491678a8a7e39b92ad5e3a7d353538867fd"} Jul 14 17:54:48.712985 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:48.712955 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:48.713315 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:48.712957 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:48.713315 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:48.713046 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:48.713315 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:48.713157 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:49.715357 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:49.715288 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:49.715770 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:49.715373 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:50.712504 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:50.712470 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:50.712719 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:50.712584 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:50.712719 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:50.712644 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:50.712841 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:50.712753 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:51.319056 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:51.319033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:51.319718 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.319143 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:51.319718 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.319189 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret podName:8b715d57-a5dc-4ab9-a089-52a1c8919e91 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:23.319175545 +0000 UTC m=+66.181965326 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret") pod "global-pull-secret-syncer-k2vhg" (UID: "8b715d57-a5dc-4ab9-a089-52a1c8919e91") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:54:51.419576 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:51.419547 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:51.419738 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.419663 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:51.419738 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.419714 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:23.41969709 +0000 UTC m=+66.282486892 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:54:51.520584 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:51.520557 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:51.520726 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.520692 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:54:51.520726 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.520705 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:54:51.520726 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.520713 2572 projected.go:196] Error preparing data for projected volume kube-api-access-k4pfm for pod openshift-network-diagnostics/network-check-target-7qd6n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:51.520818 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.520755 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm podName:f6c7e9e4-d69f-428f-b853-030128854231 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:23.520742521 +0000 UTC m=+66.383532303 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k4pfm" (UniqueName: "kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm") pod "network-check-target-7qd6n" (UID: "f6c7e9e4-d69f-428f-b853-030128854231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:54:51.712647 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:51.712553 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:51.712786 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:51.712690 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qd6n" podUID="f6c7e9e4-d69f-428f-b853-030128854231" Jul 14 17:54:51.956458 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:51.956422 2572 generic.go:356] "Generic (PLEG): container finished" podID="c264af97-795e-43eb-84e3-81aeab4aacf6" containerID="cee5ee050940da67a19d44fbddb35ce8e21a7babe08898e977b8df1ad02087fe" exitCode=0 Jul 14 17:54:51.956606 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:51.956475 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerDied","Data":"cee5ee050940da67a19d44fbddb35ce8e21a7babe08898e977b8df1ad02087fe"} Jul 14 17:54:52.712755 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.712726 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:52.713210 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.712717 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:52.713210 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:52.712861 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k2vhg" podUID="8b715d57-a5dc-4ab9-a089-52a1c8919e91" Jul 14 17:54:52.713210 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:52.712903 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:54:52.924738 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.924709 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-115.ec2.internal" event="NodeReady" Jul 14 17:54:52.924920 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.924858 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 17:54:52.960875 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.960841 2572 generic.go:356] "Generic (PLEG): container finished" podID="c264af97-795e-43eb-84e3-81aeab4aacf6" containerID="328c9e1ba32de021177e3be3ff91bac98a75f767d07e1a6b624e8176c90209c7" exitCode=0 Jul 14 17:54:52.961037 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.960909 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerDied","Data":"328c9e1ba32de021177e3be3ff91bac98a75f767d07e1a6b624e8176c90209c7"} Jul 14 17:54:52.963062 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.963041 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6db59c796c-v9grz"] Jul 14 17:54:52.966590 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.966570 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:52.969093 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.969077 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 17:54:52.969171 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.969097 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rr2k7\"" Jul 14 17:54:52.969808 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.969795 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 17:54:52.969912 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.969896 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 17:54:52.975505 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.975474 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 17:54:52.982885 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.982865 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6db59c796c-v9grz"] Jul 14 17:54:52.989571 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.989518 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x674q"] Jul 14 17:54:52.993105 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.993088 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k5bwr"] Jul 14 17:54:52.993214 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.993198 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x674q" Jul 14 17:54:52.995690 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.995667 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ljwn8\"" Jul 14 17:54:52.995690 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.995678 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 17:54:52.995842 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.995686 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 17:54:52.996504 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.996483 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:52.999749 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:52.999730 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 17:54:53.000083 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.000063 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 17:54:53.000252 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.000150 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pcnzx\"" Jul 14 17:54:53.000522 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.000503 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 17:54:53.003271 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.003250 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x674q"] Jul 14 17:54:53.003881 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.003862 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k5bwr"] Jul 14 17:54:53.132887 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.132849 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3e8c4ef-5559-4a7d-936b-8122e30e432b-config-volume\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.132887 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.132891 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2zzr\" (UniqueName: \"kubernetes.io/projected/c3e8c4ef-5559-4a7d-936b-8122e30e432b-kube-api-access-w2zzr\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.133099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.132918 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-registry-certificates\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.132937 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478be86d-41ba-406e-a434-cbc62ac47c65-cert\") pod \"ingress-canary-k5bwr\" (UID: \"478be86d-41ba-406e-a434-cbc62ac47c65\") " pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.133099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133036 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcnt9\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-kube-api-access-kcnt9\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133067 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-image-registry-private-configuration\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-trusted-ca\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133309 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133126 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-installation-pull-secrets\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133309 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133179 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-bound-sa-token\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133309 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133272 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3e8c4ef-5559-4a7d-936b-8122e30e432b-tmp-dir\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.133440 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133399 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133440 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133426 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1038199-9cee-4258-93fa-73efb2edfce6-ca-trust-extracted\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.133511 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133450 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k2w6\" (UniqueName: \"kubernetes.io/projected/478be86d-41ba-406e-a434-cbc62ac47c65-kube-api-access-7k2w6\") pod \"ingress-canary-k5bwr\" (UID: \"478be86d-41ba-406e-a434-cbc62ac47c65\") " pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.133511 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.133500 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.234823 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.234756 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2zzr\" (UniqueName: \"kubernetes.io/projected/c3e8c4ef-5559-4a7d-936b-8122e30e432b-kube-api-access-w2zzr\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.234823 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.234788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-registry-certificates\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.234823 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.234811 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478be86d-41ba-406e-a434-cbc62ac47c65-cert\") pod \"ingress-canary-k5bwr\" (UID: \"478be86d-41ba-406e-a434-cbc62ac47c65\") " pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.235040 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.234979 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcnt9\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-kube-api-access-kcnt9\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235040 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-image-registry-private-configuration\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235145 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-trusted-ca\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235145 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235073 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-installation-pull-secrets\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235145 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235104 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-bound-sa-token\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235285 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235147 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3e8c4ef-5559-4a7d-936b-8122e30e432b-tmp-dir\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.235285 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235183 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235285 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235212 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1038199-9cee-4258-93fa-73efb2edfce6-ca-trust-extracted\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235285 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235238 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k2w6\" (UniqueName: \"kubernetes.io/projected/478be86d-41ba-406e-a434-cbc62ac47c65-kube-api-access-7k2w6\") pod \"ingress-canary-k5bwr\" (UID: \"478be86d-41ba-406e-a434-cbc62ac47c65\") " pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.235285 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235267 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.235508 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.235316 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:54:53.235508 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.235331 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:54:53.235508 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.235395 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:53.735375476 +0000 UTC m=+36.598165279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:54:53.235686 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235329 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3e8c4ef-5559-4a7d-936b-8122e30e432b-config-volume\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.235889 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235746 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-registry-certificates\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.235889 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.235837 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:54:53.235889 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.235836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3e8c4ef-5559-4a7d-936b-8122e30e432b-tmp-dir\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.236082 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.235932 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:54:53.735915828 +0000 UTC m=+36.598705633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:54:53.236082 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.236011 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-trusted-ca\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.236082 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.236063 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3e8c4ef-5559-4a7d-936b-8122e30e432b-config-volume\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.236244 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.236157 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1038199-9cee-4258-93fa-73efb2edfce6-ca-trust-extracted\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.239213 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.239192 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-installation-pull-secrets\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.239308 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.239200 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478be86d-41ba-406e-a434-cbc62ac47c65-cert\") pod \"ingress-canary-k5bwr\" (UID: \"478be86d-41ba-406e-a434-cbc62ac47c65\") " pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.239308 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.239243 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-image-registry-private-configuration\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.243754 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.243727 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2zzr\" (UniqueName: \"kubernetes.io/projected/c3e8c4ef-5559-4a7d-936b-8122e30e432b-kube-api-access-w2zzr\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.243852 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.243820 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k2w6\" (UniqueName: \"kubernetes.io/projected/478be86d-41ba-406e-a434-cbc62ac47c65-kube-api-access-7k2w6\") pod \"ingress-canary-k5bwr\" (UID: \"478be86d-41ba-406e-a434-cbc62ac47c65\") " pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.244162 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.244143 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-bound-sa-token\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.244309 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.244276 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcnt9\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-kube-api-access-kcnt9\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.309926 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.309900 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k5bwr" Jul 14 17:54:53.449181 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.448866 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k5bwr"] Jul 14 17:54:53.452848 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:54:53.452820 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478be86d_41ba_406e_a434_cbc62ac47c65.slice/crio-6cecc67c64930cbb638f9976f3a11b9ad9568fcc1f87d2f868f5fc685e06a6ae WatchSource:0}: Error finding container 6cecc67c64930cbb638f9976f3a11b9ad9568fcc1f87d2f868f5fc685e06a6ae: Status 404 returned error can't find the container with id 6cecc67c64930cbb638f9976f3a11b9ad9568fcc1f87d2f868f5fc685e06a6ae Jul 14 17:54:53.713264 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.713232 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:54:53.715786 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.715766 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:54:53.716542 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.716522 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:54:53.716664 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.716555 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9r7mf\"" Jul 14 17:54:53.741084 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.741057 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:53.741184 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.741089 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:53.741230 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.741197 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:54:53.741230 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.741208 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:54:53.741230 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.741226 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:54:53.741356 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.741251 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:54:54.741234451 +0000 UTC m=+37.604024238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:54:53.741356 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:53.741277 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:54.741259145 +0000 UTC m=+37.604048942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:54:53.964507 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:53.964424 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k5bwr" event={"ID":"478be86d-41ba-406e-a434-cbc62ac47c65","Type":"ContainerStarted","Data":"6cecc67c64930cbb638f9976f3a11b9ad9568fcc1f87d2f868f5fc685e06a6ae"} Jul 14 17:54:54.712441 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.712401 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:54:54.712441 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.712440 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:54:54.714981 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.714957 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:54:54.716070 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.715844 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:54:54.716070 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.715850 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2kndq\"" Jul 14 17:54:54.751283 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.751260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:54.751412 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:54.751302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:54.751474 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:54.751410 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:54:54.751474 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:54.751451 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:54:56.751439006 +0000 UTC m=+39.614228788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:54:54.751686 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:54.751660 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:54:54.751686 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:54.751682 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:54:54.751877 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:54.751741 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:56.751724636 +0000 UTC m=+39.614514432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:54:56.766760 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:56.766713 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:54:56.767147 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:56.766774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:54:56.767147 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:56.766864 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:54:56.767147 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:56.766886 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:54:56.767147 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:56.766905 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:54:56.767147 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:56.766961 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:00.766939263 +0000 UTC m=+43.629729062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:54:56.767147 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:54:56.766977 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:55:00.766970781 +0000 UTC m=+43.629760567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:54:58.623488 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:58.623306 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:54:58.729212 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:58.729115 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:54:58.977357 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:58.977321 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k5bwr" event={"ID":"478be86d-41ba-406e-a434-cbc62ac47c65","Type":"ContainerStarted","Data":"936d44d914efc1100612f0e8307e91bffcf7e8a735f1e0a85fafc42adae4db59"} Jul 14 17:54:58.980063 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:58.979991 2572 generic.go:356] "Generic (PLEG): container finished" podID="c264af97-795e-43eb-84e3-81aeab4aacf6" containerID="2c4442519ea74cd45012dfe6614f9a916ebe8a7367b6cbdc9fe193c5193cb3ea" exitCode=0 Jul 14 17:54:58.980063 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:58.980033 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerDied","Data":"2c4442519ea74cd45012dfe6614f9a916ebe8a7367b6cbdc9fe193c5193cb3ea"} Jul 14 17:54:59.001150 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:59.001111 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k5bwr" podStartSLOduration=2.307697922 podStartE2EDuration="7.001097686s" totalImagesPullingTime="4.693399764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:58.999630452 +0000 UTC m=+41.862420247" watchObservedRunningTime="2026-07-14 17:54:59.001097686 +0000 UTC m=+41.863887489" Jul 14 17:54:59.985767 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:59.985735 2572 generic.go:356] "Generic (PLEG): container finished" podID="c264af97-795e-43eb-84e3-81aeab4aacf6" containerID="d727f5c7d08e59f2bc453ed646c1f5befdfccc173ce4c5b22aa18ebc3e33ac1c" exitCode=0 Jul 14 17:54:59.986102 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:54:59.985817 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerDied","Data":"d727f5c7d08e59f2bc453ed646c1f5befdfccc173ce4c5b22aa18ebc3e33ac1c"} Jul 14 17:55:00.798636 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:00.798571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:55:00.798636 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:00.798641 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:55:00.798838 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:00.798724 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:55:00.798838 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:00.798741 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:55:00.798838 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:00.798779 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:55:00.798838 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:00.798797 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:08.798782317 +0000 UTC m=+51.661572098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:55:00.798838 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:00.798827 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:55:08.79881322 +0000 UTC m=+51.661603038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:55:00.991361 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:00.991330 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" event={"ID":"c264af97-795e-43eb-84e3-81aeab4aacf6","Type":"ContainerStarted","Data":"4616a25fbdf3f170507d9ca1b526b338c9948e7a2c127328e320c3ff93c47bf4"} Jul 14 17:55:01.018884 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:01.018840 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nfqj2" podStartSLOduration=15.992584885 podStartE2EDuration="43.01882453s" totalImagesPullingTime="27.026239645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:01.018177494 +0000 UTC m=+43.880967276" watchObservedRunningTime="2026-07-14 17:55:01.01882453 +0000 UTC m=+43.881614333" Jul 14 17:55:01.147778 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:01.147686 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:01.248731 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:01.248678 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:03.881155 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:03.881125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:03.982472 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:03.982445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:04.529725 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:04.529690 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:04.632836 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:04.632799 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:05.539366 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:05.539330 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:05.616633 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:05.616580 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:05.826371 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:05.826336 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:05.893802 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:05.893770 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:05.995297 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:05.995251 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:06.762275 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:06.762243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:06.863891 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:06.863852 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:07.281391 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:07.281362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:07.608627 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:07.608577 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:08.860673 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:08.860637 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:55:08.860673 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:08.860674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:55:08.861107 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:08.860763 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:55:08.861107 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:08.860771 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:55:08.861107 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:08.860791 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:55:08.861107 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:08.860821 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:55:24.860808214 +0000 UTC m=+67.723598009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:55:08.861107 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:08.860842 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:24.860828384 +0000 UTC m=+67.723618168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:55:09.713219 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:09.713182 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:10.596852 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:10.596822 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:18.968737 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:18.968712 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djmwx" Jul 14 17:55:19.132826 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:19.132792 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:19.235657 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:19.235564 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:21.476155 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:21.476122 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:21.579734 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:21.579695 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:21.677226 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:21.677192 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:21.755412 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:21.755332 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:22.577115 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:22.577075 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:23.366914 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.366875 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:55:23.369929 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.369911 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:55:23.381148 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.381113 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b715d57-a5dc-4ab9-a089-52a1c8919e91-original-pull-secret\") pod \"global-pull-secret-syncer-k2vhg\" (UID: \"8b715d57-a5dc-4ab9-a089-52a1c8919e91\") " pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:55:23.438026 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.437984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:23.467978 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.467938 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:55:23.470494 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.470478 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:55:23.478700 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:23.478684 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 17:55:23.478754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:23.478737 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:56:27.478723292 +0000 UTC m=+130.341513073 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : secret "metrics-daemon-secret" not found Jul 14 17:55:23.499150 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.499120 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:23.532196 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.532169 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k2vhg" Jul 14 17:55:23.568304 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.568275 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:55:23.571016 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.570993 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:55:23.581940 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.581915 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:55:23.592835 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.592805 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pfm\" (UniqueName: \"kubernetes.io/projected/f6c7e9e4-d69f-428f-b853-030128854231-kube-api-access-k4pfm\") pod \"network-check-target-7qd6n\" (UID: \"f6c7e9e4-d69f-428f-b853-030128854231\") " pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:55:23.673311 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.673242 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k2vhg"] Jul 14 17:55:23.676653 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:55:23.676603 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b715d57_a5dc_4ab9_a089_52a1c8919e91.slice/crio-a81eabb491fb1bd767f92087f39cef06283ad6974240e2ad1def49212fc53c65 WatchSource:0}: Error finding container a81eabb491fb1bd767f92087f39cef06283ad6974240e2ad1def49212fc53c65: Status 404 returned error can't find the container with id a81eabb491fb1bd767f92087f39cef06283ad6974240e2ad1def49212fc53c65 Jul 14 17:55:23.727595 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.727563 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9r7mf\"" Jul 14 17:55:23.735869 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.735847 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:55:23.844037 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.844009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:23.858810 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.858759 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7qd6n"] Jul 14 17:55:23.862173 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:55:23.862147 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c7e9e4_d69f_428f_b853_030128854231.slice/crio-34443bd9970a17e87d26cedd9e335f54cb2695c67e9259f3c4fa12b8049977c1 WatchSource:0}: Error finding container 34443bd9970a17e87d26cedd9e335f54cb2695c67e9259f3c4fa12b8049977c1: Status 404 returned error can't find the container with id 34443bd9970a17e87d26cedd9e335f54cb2695c67e9259f3c4fa12b8049977c1 Jul 14 17:55:23.947109 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:23.947020 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:24.044176 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:24.044143 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7qd6n" event={"ID":"f6c7e9e4-d69f-428f-b853-030128854231","Type":"ContainerStarted","Data":"34443bd9970a17e87d26cedd9e335f54cb2695c67e9259f3c4fa12b8049977c1"} Jul 14 17:55:24.045168 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:24.045146 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k2vhg" event={"ID":"8b715d57-a5dc-4ab9-a089-52a1c8919e91","Type":"ContainerStarted","Data":"a81eabb491fb1bd767f92087f39cef06283ad6974240e2ad1def49212fc53c65"} Jul 14 17:55:24.879228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:24.879187 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:55:24.879766 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:24.879241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:55:24.879766 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:24.879339 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:55:24.879766 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:24.879356 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:55:24.879766 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:24.879365 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:55:24.879766 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:24.879420 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:55:56.879400431 +0000 UTC m=+99.742190212 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:55:24.879766 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:24.879439 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:56.879429174 +0000 UTC m=+99.742218955 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:55:27.054769 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:27.054729 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7qd6n" event={"ID":"f6c7e9e4-d69f-428f-b853-030128854231","Type":"ContainerStarted","Data":"7fcdeda0e3aa7607c5fae3786183f7568e0eb40cafea03086c94979aa9ed94d7"} Jul 14 17:55:27.055200 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:27.054819 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:55:27.072844 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:27.072801 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7qd6n" podStartSLOduration=66.163432592 podStartE2EDuration="1m9.072785359s" totalImagesPullingTime="2.909352767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:27.0719913 +0000 UTC m=+69.934781106" watchObservedRunningTime="2026-07-14 17:55:27.072785359 +0000 UTC m=+69.935575163" Jul 14 17:55:29.061718 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:29.061679 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k2vhg" event={"ID":"8b715d57-a5dc-4ab9-a089-52a1c8919e91","Type":"ContainerStarted","Data":"ee186fab5a99529ca33e2e9ad5cca0e7aa5c9980749de2a3c98e8e7bfd5f2e10"} Jul 14 17:55:29.084879 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:29.084832 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k2vhg" podStartSLOduration=66.354527969 podStartE2EDuration="1m11.084818101s" totalImagesPullingTime="4.730290132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:29.080597303 +0000 UTC m=+71.943387108" watchObservedRunningTime="2026-07-14 17:55:29.084818101 +0000 UTC m=+71.947607904" Jul 14 17:55:42.603484 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:42.603354 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:42.685760 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:42.685727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:43.887064 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:43.887025 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:43.986393 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:43.986366 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:53.694039 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:53.693997 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:55:56.906942 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:56.906913 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:55:56.907311 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:56.906948 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:55:56.907311 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:56.907043 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:55:56.907311 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:56.907069 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:55:56.907311 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:56.907092 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:55:56.907311 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:56.907104 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:57:00.90709032 +0000 UTC m=+163.769880102 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:55:56.907311 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:55:56.907145 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:57:00.907132211 +0000 UTC m=+163.769921993 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:55:58.060664 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:55:58.060634 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7qd6n" Jul 14 17:56:27.515029 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:56:27.514969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:56:27.515464 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:56:27.515124 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 17:56:27.515767 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:56:27.515745 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:29.515719612 +0000 UTC m=+252.378509394 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : secret "metrics-daemon-secret" not found Jul 14 17:56:55.976801 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:56:55.976759 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" Jul 14 17:56:56.004164 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:56:56.004137 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-x674q" podUID="c3e8c4ef-5559-4a7d-936b-8122e30e432b" Jul 14 17:56:56.264589 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:56:56.264508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x674q" Jul 14 17:56:56.264778 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:56:56.264508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:56:57.726281 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:56:57.726238 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:57:00.951250 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:57:00.951202 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:57:00.951250 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:57:00.951252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:57:00.951754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:57:00.951358 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:57:00.951754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:57:00.951377 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:57:00.951754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:57:00.951405 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db59c796c-v9grz: secret "image-registry-tls" not found Jul 14 17:57:00.951754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:57:00.951435 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls podName:c3e8c4ef-5559-4a7d-936b-8122e30e432b nodeName:}" failed. No retries permitted until 2026-07-14 17:59:02.951417887 +0000 UTC m=+285.814207673 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls") pod "dns-default-x674q" (UID: "c3e8c4ef-5559-4a7d-936b-8122e30e432b") : secret "dns-default-metrics-tls" not found Jul 14 17:57:00.951754 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:57:00.951461 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls podName:b1038199-9cee-4258-93fa-73efb2edfce6 nodeName:}" failed. No retries permitted until 2026-07-14 17:59:02.951443772 +0000 UTC m=+285.814233568 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls") pod "image-registry-6db59c796c-v9grz" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6") : secret "image-registry-tls" not found Jul 14 17:57:11.712642 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:57:11.712535 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:57:42.070892 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:57:42.070834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:22.249197 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.249160 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-jgsr8"] Jul 14 17:58:22.250849 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.250827 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.251770 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.251742 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz"] Jul 14 17:58:22.253670 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.253657 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.255741 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.255724 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fwhp8\"" Jul 14 17:58:22.255961 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.255942 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 17:58:22.256446 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.256429 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 17:58:22.256510 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.256431 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 17:58:22.256510 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.256430 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 17:58:22.258156 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.258005 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 17:58:22.258156 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.258059 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 17:58:22.258156 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.258124 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-g27kk\"" Jul 14 17:58:22.258364 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.258060 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:58:22.258364 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.258061 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 17:58:22.260667 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.260649 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 17:58:22.282537 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.282488 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz"] Jul 14 17:58:22.283266 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.283247 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-jgsr8"] Jul 14 17:58:22.332063 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.332037 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf"] Jul 14 17:58:22.333777 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.333763 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" Jul 14 17:58:22.336205 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.336189 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tbcpq\"" Jul 14 17:58:22.337991 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.337972 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z"] Jul 14 17:58:22.339648 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.339633 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr"] Jul 14 17:58:22.339727 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.339715 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:22.341251 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.341236 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.356918 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.356897 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 17:58:22.357004 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.356936 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 17:58:22.357056 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.357016 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:58:22.357056 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.357026 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 17:58:22.357194 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.357177 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pcp4z\"" Jul 14 17:58:22.357307 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.357292 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 17:58:22.358030 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.358015 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf"] Jul 14 17:58:22.365331 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.365248 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z"] Jul 14 17:58:22.365844 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.365829 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 14 17:58:22.365988 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.365973 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vzkt8\"" Jul 14 17:58:22.369184 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.369167 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 14 17:58:22.380440 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.380419 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr"] Jul 14 17:58:22.385159 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.385118 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:22.399506 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399484 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/81c45dce-e688-45f9-b692-672783e31d68-snapshots\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.399580 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399510 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81c45dce-e688-45f9-b692-672783e31d68-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.399580 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399531 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2rzp\" (UniqueName: \"kubernetes.io/projected/bd6d0995-b613-4d4d-9078-95a2d388fcc4-kube-api-access-z2rzp\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.399580 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399552 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81c45dce-e688-45f9-b692-672783e31d68-tmp\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.399580 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399570 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81c45dce-e688-45f9-b692-672783e31d68-serving-cert\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.399726 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399601 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd6d0995-b613-4d4d-9078-95a2d388fcc4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.399726 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399676 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd6d0995-b613-4d4d-9078-95a2d388fcc4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.399726 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399696 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl89k\" (UniqueName: \"kubernetes.io/projected/81c45dce-e688-45f9-b692-672783e31d68-kube-api-access-fl89k\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.399814 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.399729 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81c45dce-e688-45f9-b692-672783e31d68-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501066 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.500985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/81c45dce-e688-45f9-b692-672783e31d68-snapshots\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501066 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501021 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7lsj\" (UniqueName: \"kubernetes.io/projected/ef9e043e-0000-4fd3-a54d-a920137226c3-kube-api-access-f7lsj\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:22.501066 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81c45dce-e688-45f9-b692-672783e31d68-tmp\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501339 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81c45dce-e688-45f9-b692-672783e31d68-serving-cert\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501339 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501125 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:22.501339 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd6d0995-b613-4d4d-9078-95a2d388fcc4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.501339 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501241 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55ww\" (UniqueName: \"kubernetes.io/projected/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-kube-api-access-n55ww\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.501339 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81c45dce-e688-45f9-b692-672783e31d68-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501339 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501311 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzvwc\" (UniqueName: \"kubernetes.io/projected/f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c-kube-api-access-zzvwc\") pod \"network-check-source-869dcfcdcd-n2wvf\" (UID: \"f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" Jul 14 17:58:22.501636 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2rzp\" (UniqueName: \"kubernetes.io/projected/bd6d0995-b613-4d4d-9078-95a2d388fcc4-kube-api-access-z2rzp\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.501636 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501365 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.501636 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd6d0995-b613-4d4d-9078-95a2d388fcc4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.501857 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501672 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81c45dce-e688-45f9-b692-672783e31d68-tmp\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501857 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501718 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/81c45dce-e688-45f9-b692-672783e31d68-snapshots\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501857 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl89k\" (UniqueName: \"kubernetes.io/projected/81c45dce-e688-45f9-b692-672783e31d68-kube-api-access-fl89k\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501857 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501806 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81c45dce-e688-45f9-b692-672783e31d68-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.501857 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501829 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd6d0995-b613-4d4d-9078-95a2d388fcc4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.502099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.501951 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.502317 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.502297 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81c45dce-e688-45f9-b692-672783e31d68-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.502387 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.502351 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81c45dce-e688-45f9-b692-672783e31d68-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.503579 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.503561 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81c45dce-e688-45f9-b692-672783e31d68-serving-cert\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.503718 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.503702 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd6d0995-b613-4d4d-9078-95a2d388fcc4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.506137 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.506099 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:22.510116 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.510094 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2rzp\" (UniqueName: \"kubernetes.io/projected/bd6d0995-b613-4d4d-9078-95a2d388fcc4-kube-api-access-z2rzp\") pod \"kube-storage-version-migrator-operator-799cb7f958-qszdz\" (UID: \"bd6d0995-b613-4d4d-9078-95a2d388fcc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.510204 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.510134 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl89k\" (UniqueName: \"kubernetes.io/projected/81c45dce-e688-45f9-b692-672783e31d68-kube-api-access-fl89k\") pod \"insights-operator-68d9dd94cf-jgsr8\" (UID: \"81c45dce-e688-45f9-b692-672783e31d68\") " pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.561335 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.561297 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" Jul 14 17:58:22.565042 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.565018 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" Jul 14 17:58:22.603106 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.603064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:22.603293 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.603162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n55ww\" (UniqueName: \"kubernetes.io/projected/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-kube-api-access-n55ww\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.603293 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.603204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzvwc\" (UniqueName: \"kubernetes.io/projected/f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c-kube-api-access-zzvwc\") pod \"network-check-source-869dcfcdcd-n2wvf\" (UID: \"f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" Jul 14 17:58:22.603293 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:22.603233 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:58:22.603452 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.603236 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.603452 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:22.603314 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls podName:ef9e043e-0000-4fd3-a54d-a920137226c3 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:23.103295062 +0000 UTC m=+245.966084862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls") pod "cluster-samples-operator-65db858894-nfr5z" (UID: "ef9e043e-0000-4fd3-a54d-a920137226c3") : secret "samples-operator-tls" not found Jul 14 17:58:22.603452 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.603435 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.603601 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.603486 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7lsj\" (UniqueName: \"kubernetes.io/projected/ef9e043e-0000-4fd3-a54d-a920137226c3-kube-api-access-f7lsj\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:22.603601 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:22.603547 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:22.603601 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:22.603598 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls podName:57e2a584-e4f2-4c7b-ad1c-c19bed20c47f nodeName:}" failed. No retries permitted until 2026-07-14 17:58:23.103584548 +0000 UTC m=+245.966374347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dp9sr" (UID: "57e2a584-e4f2-4c7b-ad1c-c19bed20c47f") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:22.604066 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.604042 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.606030 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.606005 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:22.612954 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.612907 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7lsj\" (UniqueName: \"kubernetes.io/projected/ef9e043e-0000-4fd3-a54d-a920137226c3-kube-api-access-f7lsj\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:22.616056 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.615346 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzvwc\" (UniqueName: \"kubernetes.io/projected/f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c-kube-api-access-zzvwc\") pod \"network-check-source-869dcfcdcd-n2wvf\" (UID: \"f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" Jul 14 17:58:22.616435 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.616231 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55ww\" (UniqueName: \"kubernetes.io/projected/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-kube-api-access-n55ww\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:22.642447 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.642415 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" Jul 14 17:58:22.702640 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.702579 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-jgsr8"] Jul 14 17:58:22.703678 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.703595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:22.707814 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:58:22.707777 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c45dce_e688_45f9_b692_672783e31d68.slice/crio-293417d77382dfefffa4b58fac359f0c7d5b61f095a325fd3981aa2675af1cc5 WatchSource:0}: Error finding container 293417d77382dfefffa4b58fac359f0c7d5b61f095a325fd3981aa2675af1cc5: Status 404 returned error can't find the container with id 293417d77382dfefffa4b58fac359f0c7d5b61f095a325fd3981aa2675af1cc5 Jul 14 17:58:22.718212 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.718191 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz"] Jul 14 17:58:22.721180 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:58:22.721159 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6d0995_b613_4d4d_9078_95a2d388fcc4.slice/crio-3d76d04b715f4eb4b5fc1f45593e82e2c3799fda51f4d7ecb9d4370c8a4583b4 WatchSource:0}: Error finding container 3d76d04b715f4eb4b5fc1f45593e82e2c3799fda51f4d7ecb9d4370c8a4583b4: Status 404 returned error can't find the container with id 3d76d04b715f4eb4b5fc1f45593e82e2c3799fda51f4d7ecb9d4370c8a4583b4 Jul 14 17:58:22.775330 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:22.775259 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf"] Jul 14 17:58:22.778300 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:58:22.778268 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d00b2f_4d67_4a88_8ca9_34a2a3808d0c.slice/crio-4727f52d31067cb932a88366b90de66080a23d2e4b6537fd897e0ea879bb0a85 WatchSource:0}: Error finding container 4727f52d31067cb932a88366b90de66080a23d2e4b6537fd897e0ea879bb0a85: Status 404 returned error can't find the container with id 4727f52d31067cb932a88366b90de66080a23d2e4b6537fd897e0ea879bb0a85 Jul 14 17:58:23.109099 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.109072 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:23.109270 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.109154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:23.109270 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:23.109221 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:58:23.109270 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:23.109237 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:23.109422 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:23.109283 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls podName:57e2a584-e4f2-4c7b-ad1c-c19bed20c47f nodeName:}" failed. No retries permitted until 2026-07-14 17:58:24.109268791 +0000 UTC m=+246.972058572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dp9sr" (UID: "57e2a584-e4f2-4c7b-ad1c-c19bed20c47f") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:23.109422 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:23.109300 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls podName:ef9e043e-0000-4fd3-a54d-a920137226c3 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:24.109293423 +0000 UTC m=+246.972083205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls") pod "cluster-samples-operator-65db858894-nfr5z" (UID: "ef9e043e-0000-4fd3-a54d-a920137226c3") : secret "samples-operator-tls" not found Jul 14 17:58:23.470714 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.470565 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" event={"ID":"81c45dce-e688-45f9-b692-672783e31d68","Type":"ContainerStarted","Data":"293417d77382dfefffa4b58fac359f0c7d5b61f095a325fd3981aa2675af1cc5"} Jul 14 17:58:23.472317 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.472287 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" event={"ID":"bd6d0995-b613-4d4d-9078-95a2d388fcc4","Type":"ContainerStarted","Data":"3d76d04b715f4eb4b5fc1f45593e82e2c3799fda51f4d7ecb9d4370c8a4583b4"} Jul 14 17:58:23.474731 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.474666 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" event={"ID":"f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c","Type":"ContainerStarted","Data":"68d1603eca3ccfd4d8e61fbaa99f2c2880836916c2a9c822c8175bf48592e6a6"} Jul 14 17:58:23.474731 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.474708 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" event={"ID":"f7d00b2f-4d67-4a88-8ca9-34a2a3808d0c","Type":"ContainerStarted","Data":"4727f52d31067cb932a88366b90de66080a23d2e4b6537fd897e0ea879bb0a85"} Jul 14 17:58:23.493579 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:23.493462 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-n2wvf" podStartSLOduration=1.493444894 podStartE2EDuration="1.493444894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:23.492798986 +0000 UTC m=+246.355588791" watchObservedRunningTime="2026-07-14 17:58:23.493444894 +0000 UTC m=+246.356234678" Jul 14 17:58:24.118015 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:24.117969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:24.118214 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:24.118048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:24.118214 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:24.118153 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:24.118339 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:24.118212 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:58:24.118339 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:24.118224 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls podName:57e2a584-e4f2-4c7b-ad1c-c19bed20c47f nodeName:}" failed. No retries permitted until 2026-07-14 17:58:26.118209718 +0000 UTC m=+248.980999499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dp9sr" (UID: "57e2a584-e4f2-4c7b-ad1c-c19bed20c47f") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:24.118339 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:24.118272 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls podName:ef9e043e-0000-4fd3-a54d-a920137226c3 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:26.118257103 +0000 UTC m=+248.981046889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls") pod "cluster-samples-operator-65db858894-nfr5z" (UID: "ef9e043e-0000-4fd3-a54d-a920137226c3") : secret "samples-operator-tls" not found Jul 14 17:58:25.482165 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:25.482131 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" event={"ID":"81c45dce-e688-45f9-b692-672783e31d68","Type":"ContainerStarted","Data":"bff8251b2664954e6ef82a4b3f196ede55dc0909843264d577497a862c5b52f7"} Jul 14 17:58:25.484045 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:25.484011 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" event={"ID":"bd6d0995-b613-4d4d-9078-95a2d388fcc4","Type":"ContainerStarted","Data":"17fc6913936130288c572738b7842afe4b39d72d8e415e812651bbb3c0e8cdc2"} Jul 14 17:58:25.500005 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:25.499961 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" podStartSLOduration=0.849840325 podStartE2EDuration="3.499944829s" totalImagesPullingTime="2.650104504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:25.499183006 +0000 UTC m=+248.361972809" watchObservedRunningTime="2026-07-14 17:58:25.499944829 +0000 UTC m=+248.362734632" Jul 14 17:58:25.516125 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:25.516064 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" podStartSLOduration=0.878102496 podStartE2EDuration="3.516042709s" totalImagesPullingTime="2.637940213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:25.515219739 +0000 UTC m=+248.378009544" watchObservedRunningTime="2026-07-14 17:58:25.516042709 +0000 UTC m=+248.378832515" Jul 14 17:58:26.138475 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:26.138433 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:26.138670 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:26.138500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:26.138670 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:26.138585 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:26.138670 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:26.138642 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:58:26.138853 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:26.138674 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls podName:57e2a584-e4f2-4c7b-ad1c-c19bed20c47f nodeName:}" failed. No retries permitted until 2026-07-14 17:58:30.13865749 +0000 UTC m=+253.001447276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dp9sr" (UID: "57e2a584-e4f2-4c7b-ad1c-c19bed20c47f") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:26.138853 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:26.138696 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls podName:ef9e043e-0000-4fd3-a54d-a920137226c3 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:30.138680555 +0000 UTC m=+253.001470346 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls") pod "cluster-samples-operator-65db858894-nfr5z" (UID: "ef9e043e-0000-4fd3-a54d-a920137226c3") : secret "samples-operator-tls" not found Jul 14 17:58:26.380540 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:26.380507 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:26.449928 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:26.449828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:26.515833 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:26.515794 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:26.965134 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:26.965100 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:27.028672 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.028637 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:27.198381 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.198348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:27.305138 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.305084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:27.307936 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.307908 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-njpf6"] Jul 14 17:58:27.310674 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.310651 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.314465 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.314435 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5r9n7\"" Jul 14 17:58:27.314731 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.314710 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 17:58:27.315228 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.315207 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 17:58:27.326425 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.326405 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-njpf6"] Jul 14 17:58:27.388829 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.388785 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:27.451203 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.451173 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/015e5b9a-80c0-472a-b3f7-86de6814895c-data-volume\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.451358 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.451205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.451358 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.451288 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9vd\" (UniqueName: \"kubernetes.io/projected/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-api-access-xw9vd\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.451358 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.451337 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/015e5b9a-80c0-472a-b3f7-86de6814895c-crio-socket\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.451504 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.451395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552638 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.552589 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/015e5b9a-80c0-472a-b3f7-86de6814895c-crio-socket\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.552667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.552715 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/015e5b9a-80c0-472a-b3f7-86de6814895c-crio-socket\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.552727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/015e5b9a-80c0-472a-b3f7-86de6814895c-data-volume\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.552754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:27.552785 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.552807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9vd\" (UniqueName: \"kubernetes.io/projected/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-api-access-xw9vd\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:27.552850 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:28.052835457 +0000 UTC m=+250.915625239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : configmap "kube-rbac-proxy" not found Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:27.552886 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:58:27.552978 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:27.552939 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:28.052924019 +0000 UTC m=+250.915713804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : secret "insights-runtime-extractor-tls" not found Jul 14 17:58:27.553284 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.553075 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/015e5b9a-80c0-472a-b3f7-86de6814895c-data-volume\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.563662 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.563585 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9vd\" (UniqueName: \"kubernetes.io/projected/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-api-access-xw9vd\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:27.685953 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.685918 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:27.784520 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:27.784485 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:28.056869 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:28.056834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:28.057049 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:28.056895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:28.057049 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:28.056963 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:58:28.057049 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:28.056987 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:58:28.057049 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:28.057022 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:29.057007592 +0000 UTC m=+251.919797374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : configmap "kube-rbac-proxy" not found Jul 14 17:58:28.057049 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:28.057040 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:29.057028644 +0000 UTC m=+251.919818426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : secret "insights-runtime-extractor-tls" not found Jul 14 17:58:29.065941 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:29.065902 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:29.066415 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:29.065964 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:29.066415 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:29.066029 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:58:29.066415 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:29.066094 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:58:29.066415 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:29.066118 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:31.066097465 +0000 UTC m=+253.928887263 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : configmap "kube-rbac-proxy" not found Jul 14 17:58:29.066415 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:29.066136 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:31.0661257 +0000 UTC m=+253.928915482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : secret "insights-runtime-extractor-tls" not found Jul 14 17:58:29.384096 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:29.384016 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:29.484059 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:29.484026 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:29.569004 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:29.568968 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:58:29.569169 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:29.569112 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 17:58:29.569213 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:29.569179 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs podName:ae5d25a0-c8fc-4856-89f8-e47ac9945fa1 nodeName:}" failed. No retries permitted until 2026-07-14 18:00:31.569164145 +0000 UTC m=+374.431953927 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs") pod "network-metrics-daemon-kmzpv" (UID: "ae5d25a0-c8fc-4856-89f8-e47ac9945fa1") : secret "metrics-daemon-secret" not found Jul 14 17:58:30.105825 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:30.105796 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5q98w_03a7cf65-9266-4371-8921-8568aeffe9d2/dns-node-resolver/0.log" Jul 14 17:58:30.173033 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:30.172996 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:30.173220 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:30.173058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:30.173220 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:30.173130 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:30.173220 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:30.173175 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:58:30.173220 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:30.173189 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls podName:57e2a584-e4f2-4c7b-ad1c-c19bed20c47f nodeName:}" failed. No retries permitted until 2026-07-14 17:58:38.173175757 +0000 UTC m=+261.035965539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dp9sr" (UID: "57e2a584-e4f2-4c7b-ad1c-c19bed20c47f") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:30.173220 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:30.173221 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls podName:ef9e043e-0000-4fd3-a54d-a920137226c3 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:38.173206988 +0000 UTC m=+261.035996770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls") pod "cluster-samples-operator-65db858894-nfr5z" (UID: "ef9e043e-0000-4fd3-a54d-a920137226c3") : secret "samples-operator-tls" not found Jul 14 17:58:30.349003 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:30.348972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:30.900439 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:30.900411 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-brnmt_b55e97e6-bdd1-435b-8a18-1975c8640be7/node-ca/0.log" Jul 14 17:58:31.078811 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:31.078779 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:31.078956 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:31.078843 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:31.078956 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:31.078895 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:58:31.079024 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:31.078958 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:35.078942673 +0000 UTC m=+257.941732454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : configmap "kube-rbac-proxy" not found Jul 14 17:58:31.079024 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:31.078960 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:58:31.079024 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:31.078997 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:35.078986915 +0000 UTC m=+257.941776698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : secret "insights-runtime-extractor-tls" not found Jul 14 17:58:31.900947 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:31.900910 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k5bwr_478be86d-41ba-406e-a434-cbc62ac47c65/serve-healthcheck-canary/0.log" Jul 14 17:58:32.517382 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:32.517348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:32.617992 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:32.617964 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:32.704381 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:32.704350 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-qszdz_bd6d0995-b613-4d4d-9078-95a2d388fcc4/kube-storage-version-migrator-operator/0.log" Jul 14 17:58:32.904210 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:32.904124 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:33.025755 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:33.025725 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:33.127175 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:33.127142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:35.110376 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:35.110344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:35.110761 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:35.110423 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:35.110761 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:35.110491 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:58:35.110761 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:35.110549 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:43.110535907 +0000 UTC m=+265.973325693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : secret "insights-runtime-extractor-tls" not found Jul 14 17:58:35.110761 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:35.110494 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:58:35.110761 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:35.110658 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm podName:015e5b9a-80c0-472a-b3f7-86de6814895c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:43.110643925 +0000 UTC m=+265.973433707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njpf6" (UID: "015e5b9a-80c0-472a-b3f7-86de6814895c") : configmap "kube-rbac-proxy" not found Jul 14 17:58:35.389405 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:35.389321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:35.455814 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:35.455787 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:35.814403 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:35.814373 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:36.292109 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:36.292023 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:36.519906 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:36.519874 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:36.622172 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:36.622138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:36.904118 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:36.904027 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:37.090481 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:37.090451 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:37.422765 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:37.422728 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:38.238361 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.238328 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:38.238522 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.238376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:38.238522 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:38.238488 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:38.238597 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:38.238562 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls podName:57e2a584-e4f2-4c7b-ad1c-c19bed20c47f nodeName:}" failed. No retries permitted until 2026-07-14 17:58:54.238546681 +0000 UTC m=+277.101336462 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-dp9sr" (UID: "57e2a584-e4f2-4c7b-ad1c-c19bed20c47f") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:58:38.240792 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.240758 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:38.241033 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.241015 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef9e043e-0000-4fd3-a54d-a920137226c3-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nfr5z\" (UID: \"ef9e043e-0000-4fd3-a54d-a920137226c3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:38.250329 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.250308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" Jul 14 17:58:38.378442 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.378407 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z"] Jul 14 17:58:38.428597 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.428535 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:38.526222 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.526161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" event={"ID":"ef9e043e-0000-4fd3-a54d-a920137226c3","Type":"ContainerStarted","Data":"f7c6abeba90786cf29351701dcb320c9ee6e4f9c0ba28da2f82d938622b79ab8"} Jul 14 17:58:38.631728 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.631656 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:38.735080 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:38.735047 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:39.010575 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:39.010545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:39.510784 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:39.510749 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:39.811227 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:39.811195 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:40.018651 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:40.018598 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:40.423952 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:40.423917 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:40.625179 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:40.625142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:41.221889 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:41.221853 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:41.536369 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:41.536292 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" event={"ID":"ef9e043e-0000-4fd3-a54d-a920137226c3","Type":"ContainerStarted","Data":"6ee69ec2f23ef243cd5bec9350b0e5412b040581fd838514cd063bb5a52a35f0"} Jul 14 17:58:41.536369 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:41.536339 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" event={"ID":"ef9e043e-0000-4fd3-a54d-a920137226c3","Type":"ContainerStarted","Data":"9c789174d3d041cbd1a325e1328aff0e42ad14f95c60ac38e3827e9640b7e4cf"} Jul 14 17:58:41.554373 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:41.554328 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nfr5z" podStartSLOduration=17.40172988 podStartE2EDuration="19.554313294s" totalImagesPullingTime="2.152583414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:41.553349241 +0000 UTC m=+264.416139046" watchObservedRunningTime="2026-07-14 17:58:41.554313294 +0000 UTC m=+264.417103136" Jul 14 17:58:43.177922 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.177885 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:43.178387 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.177981 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:43.178575 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.178553 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/015e5b9a-80c0-472a-b3f7-86de6814895c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:43.180291 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.180270 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/015e5b9a-80c0-472a-b3f7-86de6814895c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njpf6\" (UID: \"015e5b9a-80c0-472a-b3f7-86de6814895c\") " pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:43.222165 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.222141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-njpf6" Jul 14 17:58:43.346855 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.346823 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-njpf6"] Jul 14 17:58:43.349995 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:58:43.349969 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015e5b9a_80c0_472a_b3f7_86de6814895c.slice/crio-04669c8589dca07c8c795ab3a37033bf3f9b6d1ff4fb4f793224dd43a0a5a3ab WatchSource:0}: Error finding container 04669c8589dca07c8c795ab3a37033bf3f9b6d1ff4fb4f793224dd43a0a5a3ab: Status 404 returned error can't find the container with id 04669c8589dca07c8c795ab3a37033bf3f9b6d1ff4fb4f793224dd43a0a5a3ab Jul 14 17:58:43.541663 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.541574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:43.543657 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.543631 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njpf6" event={"ID":"015e5b9a-80c0-472a-b3f7-86de6814895c","Type":"ContainerStarted","Data":"04669c8589dca07c8c795ab3a37033bf3f9b6d1ff4fb4f793224dd43a0a5a3ab"} Jul 14 17:58:43.643253 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:43.643225 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:44.017821 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:44.017790 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:47.563902 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:47.563868 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njpf6" event={"ID":"015e5b9a-80c0-472a-b3f7-86de6814895c","Type":"ContainerStarted","Data":"4a3c407109cedc0c70f09309230a594483a13ad64d5b3be8bad492be0e248a4e"} Jul 14 17:58:47.855925 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:47.855837 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:47.957091 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:47.957067 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:50.162929 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:50.162899 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:50.263415 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:50.263375 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:50.365251 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:50.365165 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:50.576964 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:50.576927 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njpf6" event={"ID":"015e5b9a-80c0-472a-b3f7-86de6814895c","Type":"ContainerStarted","Data":"a8c1cf7b889b81e5e3ac34be05f7cc39be0588721581ea994ab5008ff34f80c6"} Jul 14 17:58:50.979580 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:50.979456 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:51.050603 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:51.050565 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:52.585149 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:52.585116 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njpf6" event={"ID":"015e5b9a-80c0-472a-b3f7-86de6814895c","Type":"ContainerStarted","Data":"0fc84ad4291de560b30eb80fb6a0ddf037aeae6b1d8f66c837becd65dac864db"} Jul 14 17:58:52.606138 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:52.606098 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-njpf6" podStartSLOduration=16.789752652 podStartE2EDuration="25.606084236s" totalImagesPullingTime="8.816331584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:58:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:52.604810175 +0000 UTC m=+275.467599979" watchObservedRunningTime="2026-07-14 17:58:52.606084236 +0000 UTC m=+275.468874040" Jul 14 17:58:54.268180 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:54.268146 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:54.270663 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:54.270644 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/57e2a584-e4f2-4c7b-ad1c-c19bed20c47f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-dp9sr\" (UID: \"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:54.454496 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:54.454465 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" Jul 14 17:58:54.577717 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:54.577688 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr"] Jul 14 17:58:54.581244 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:58:54.581220 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e2a584_e4f2_4c7b_ad1c_c19bed20c47f.slice/crio-73cadbf8f3b7eda1441449a3458426cd11994defdca8612e8ae06822d686a201 WatchSource:0}: Error finding container 73cadbf8f3b7eda1441449a3458426cd11994defdca8612e8ae06822d686a201: Status 404 returned error can't find the container with id 73cadbf8f3b7eda1441449a3458426cd11994defdca8612e8ae06822d686a201 Jul 14 17:58:54.591847 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:54.591822 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" event={"ID":"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f","Type":"ContainerStarted","Data":"73cadbf8f3b7eda1441449a3458426cd11994defdca8612e8ae06822d686a201"} Jul 14 17:58:56.884760 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:56.884671 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:56.986154 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:56.986124 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:57.086009 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:57.085975 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:57.231140 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:57.231039 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:58:57.601832 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:57.601803 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" event={"ID":"57e2a584-e4f2-4c7b-ad1c-c19bed20c47f","Type":"ContainerStarted","Data":"2eb8b986bc338ab4953aa6e5922c8edb36472fd4f21f845fdabc4375b14692e1"} Jul 14 17:58:57.618984 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:57.618938 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-dp9sr" podStartSLOduration=33.621995582 podStartE2EDuration="35.618922112s" totalImagesPullingTime="1.99692653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:57.618890231 +0000 UTC m=+280.481680036" watchObservedRunningTime="2026-07-14 17:58:57.618922112 +0000 UTC m=+280.481711919" Jul 14 17:58:59.265852 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:59.265806 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" Jul 14 17:58:59.265852 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:58:59.265806 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-x674q" podUID="c3e8c4ef-5559-4a7d-936b-8122e30e432b" Jul 14 17:58:59.611401 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:59.611373 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x674q" Jul 14 17:58:59.611571 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:58:59.611373 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:59:03.036949 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.036905 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:59:03.036949 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.036953 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:59:03.039383 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.039356 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3e8c4ef-5559-4a7d-936b-8122e30e432b-metrics-tls\") pod \"dns-default-x674q\" (UID: \"c3e8c4ef-5559-4a7d-936b-8122e30e432b\") " pod="openshift-dns/dns-default-x674q" Jul 14 17:59:03.039597 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.039578 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"image-registry-6db59c796c-v9grz\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:59:03.214802 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.214769 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rr2k7\"" Jul 14 17:59:03.215357 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.215341 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ljwn8\"" Jul 14 17:59:03.222382 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.222357 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:59:03.222476 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.222385 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x674q" Jul 14 17:59:03.378253 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.378216 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6db59c796c-v9grz"] Jul 14 17:59:03.382542 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:59:03.382479 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1038199_9cee_4258_93fa_73efb2edfce6.slice/crio-e441c59ca3a36fc5b286be288fb624b30fbcee23ba3b5db8fdc5f9d0c1b4ea43 WatchSource:0}: Error finding container e441c59ca3a36fc5b286be288fb624b30fbcee23ba3b5db8fdc5f9d0c1b4ea43: Status 404 returned error can't find the container with id e441c59ca3a36fc5b286be288fb624b30fbcee23ba3b5db8fdc5f9d0c1b4ea43 Jul 14 17:59:03.383163 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.383147 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x674q"] Jul 14 17:59:03.386151 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:59:03.386131 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e8c4ef_5559_4a7d_936b_8122e30e432b.slice/crio-c1c5c4bc73dfe45e843eafce8390a6672c8e8e449c5da75d5d5b1f783156a07d WatchSource:0}: Error finding container c1c5c4bc73dfe45e843eafce8390a6672c8e8e449c5da75d5d5b1f783156a07d: Status 404 returned error can't find the container with id c1c5c4bc73dfe45e843eafce8390a6672c8e8e449c5da75d5d5b1f783156a07d Jul 14 17:59:03.624578 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.624491 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" event={"ID":"b1038199-9cee-4258-93fa-73efb2edfce6","Type":"ContainerStarted","Data":"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489"} Jul 14 17:59:03.624578 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.624547 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" event={"ID":"b1038199-9cee-4258-93fa-73efb2edfce6","Type":"ContainerStarted","Data":"e441c59ca3a36fc5b286be288fb624b30fbcee23ba3b5db8fdc5f9d0c1b4ea43"} Jul 14 17:59:03.624578 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.624569 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:59:03.625572 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.625546 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x674q" event={"ID":"c3e8c4ef-5559-4a7d-936b-8122e30e432b","Type":"ContainerStarted","Data":"c1c5c4bc73dfe45e843eafce8390a6672c8e8e449c5da75d5d5b1f783156a07d"} Jul 14 17:59:03.669107 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:03.669061 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podStartSLOduration=285.669045522 podStartE2EDuration="4m45.669045522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:03.667926613 +0000 UTC m=+286.530716428" watchObservedRunningTime="2026-07-14 17:59:03.669045522 +0000 UTC m=+286.531835320" Jul 14 17:59:05.633090 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:05.633052 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x674q" event={"ID":"c3e8c4ef-5559-4a7d-936b-8122e30e432b","Type":"ContainerStarted","Data":"5b8bdefd8c6bbd1474a829f080050b26cff1b6d2e7b8e80570b47e66a5c53c89"} Jul 14 17:59:05.633472 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:05.633145 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x674q" event={"ID":"c3e8c4ef-5559-4a7d-936b-8122e30e432b","Type":"ContainerStarted","Data":"35295dda463ae50d5e771ed71973e24e27c942e98f902bb2883eb67f9f05370a"} Jul 14 17:59:05.633472 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:05.633175 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x674q" Jul 14 17:59:05.652925 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:05.652881 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x674q" podStartSLOduration=251.751249982 podStartE2EDuration="4m13.652866621s" totalImagesPullingTime="1.901616639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:05.651679299 +0000 UTC m=+288.514469103" watchObservedRunningTime="2026-07-14 17:59:05.652866621 +0000 UTC m=+288.515656425" Jul 14 17:59:06.563949 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:06.563913 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:06.665635 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:06.665584 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:11.827865 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:11.827825 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:11.891948 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:11.891915 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:12.581262 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:12.581222 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:12.638722 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:12.638695 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:14.713276 ip-10-0-129-115 kubenswrapper[2572]: E0714 17:59:14.713235 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kmzpv" podUID="ae5d25a0-c8fc-4856-89f8-e47ac9945fa1" Jul 14 17:59:15.639214 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:15.639188 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x674q" Jul 14 17:59:16.395749 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:16.395718 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:16.463439 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:16.463414 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:17.562397 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:17.562374 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 17:59:23.226794 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:23.226760 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:23.245124 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:23.226810 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:24.632893 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:24.632857 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:24.633243 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:24.632905 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:25.953508 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:25.953468 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:26.712862 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:26.712836 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 17:59:27.435127 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:27.435080 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:31.729231 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:31.729202 2572 generic.go:356] "Generic (PLEG): container finished" podID="81c45dce-e688-45f9-b692-672783e31d68" containerID="bff8251b2664954e6ef82a4b3f196ede55dc0909843264d577497a862c5b52f7" exitCode=0 Jul 14 17:59:31.729634 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:31.729274 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" event={"ID":"81c45dce-e688-45f9-b692-672783e31d68","Type":"ContainerDied","Data":"bff8251b2664954e6ef82a4b3f196ede55dc0909843264d577497a862c5b52f7"} Jul 14 17:59:31.729634 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:31.729554 2572 scope.go:117] "RemoveContainer" containerID="bff8251b2664954e6ef82a4b3f196ede55dc0909843264d577497a862c5b52f7" Jul 14 17:59:32.734485 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:32.734454 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jgsr8" event={"ID":"81c45dce-e688-45f9-b692-672783e31d68","Type":"ContainerStarted","Data":"e76220b59fec65f92aff42fc6618fb35ad41e7122ad88befa939013cb7cdf121"} Jul 14 17:59:33.226987 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:33.226958 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:33.227146 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:33.227004 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:34.632309 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:34.632275 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:34.632763 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:34.632327 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:37.621156 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:37.621118 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:37.699884 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:37.699860 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:38.127020 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:38.126984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:43.227192 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:43.227162 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:43.227662 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:43.227217 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:43.227662 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:43.227264 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 17:59:43.227815 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:43.227780 2572 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489"} pod="openshift-image-registry/image-registry-6db59c796c-v9grz" containerMessage="Container registry failed liveness probe, will be restarted" Jul 14 17:59:43.231431 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:43.231403 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:43.231546 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:43.231446 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:45.077291 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:45.077248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:45.177820 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:45.177783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:53.231487 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:53.231456 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 17:59:53.231876 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:53.231518 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:54.423779 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:54.423740 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:54.519773 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:54.519738 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:54.618248 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:54.618214 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:54.729579 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:54.729491 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:54.819314 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:54.819279 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:56.806713 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:56.806682 2572 generic.go:356] "Generic (PLEG): container finished" podID="bd6d0995-b613-4d4d-9078-95a2d388fcc4" containerID="17fc6913936130288c572738b7842afe4b39d72d8e415e812651bbb3c0e8cdc2" exitCode=0 Jul 14 17:59:56.807074 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:56.806752 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" event={"ID":"bd6d0995-b613-4d4d-9078-95a2d388fcc4","Type":"ContainerDied","Data":"17fc6913936130288c572738b7842afe4b39d72d8e415e812651bbb3c0e8cdc2"} Jul 14 17:59:56.807074 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:56.807050 2572 scope.go:117] "RemoveContainer" containerID="17fc6913936130288c572738b7842afe4b39d72d8e415e812651bbb3c0e8cdc2" Jul 14 17:59:57.066164 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.066087 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:57.111941 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.111915 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-hxmn6"] Jul 14 17:59:57.115369 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.115354 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.118175 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.118154 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 14 17:59:57.118299 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.118180 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 14 17:59:57.118299 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.118191 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wqhbs\"" Jul 14 17:59:57.118926 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.118911 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 17:59:57.125954 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.125934 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-hxmn6"] Jul 14 17:59:57.138325 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.138302 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:57.205392 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.205360 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xrc8\" (UniqueName: \"kubernetes.io/projected/5f1dac4a-ae12-46d7-a951-3d89dd37f481-kube-api-access-6xrc8\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.205392 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.205395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f1dac4a-ae12-46d7-a951-3d89dd37f481-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.205602 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.205438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dac4a-ae12-46d7-a951-3d89dd37f481-metrics-client-ca\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.205602 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.205476 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f1dac4a-ae12-46d7-a951-3d89dd37f481-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.263346 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.263313 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 17:59:57.306382 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.306362 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dac4a-ae12-46d7-a951-3d89dd37f481-metrics-client-ca\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.306471 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.306403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f1dac4a-ae12-46d7-a951-3d89dd37f481-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.306471 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.306448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xrc8\" (UniqueName: \"kubernetes.io/projected/5f1dac4a-ae12-46d7-a951-3d89dd37f481-kube-api-access-6xrc8\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.306557 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.306480 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f1dac4a-ae12-46d7-a951-3d89dd37f481-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.307204 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.307143 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dac4a-ae12-46d7-a951-3d89dd37f481-metrics-client-ca\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.309134 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.309111 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f1dac4a-ae12-46d7-a951-3d89dd37f481-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.309202 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.309115 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f1dac4a-ae12-46d7-a951-3d89dd37f481-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.321117 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.321067 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xrc8\" (UniqueName: \"kubernetes.io/projected/5f1dac4a-ae12-46d7-a951-3d89dd37f481-kube-api-access-6xrc8\") pod \"prometheus-operator-64785b6675-hxmn6\" (UID: \"5f1dac4a-ae12-46d7-a951-3d89dd37f481\") " pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.424221 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.424187 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" Jul 14 17:59:57.555366 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.555212 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-hxmn6"] Jul 14 17:59:57.557754 ip-10-0-129-115 kubenswrapper[2572]: W0714 17:59:57.557733 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1dac4a_ae12_46d7_a951_3d89dd37f481.slice/crio-e24d870b889aac7b90e57a8f86cde6ffdd14724a4deac1a0881f41af5d391346 WatchSource:0}: Error finding container e24d870b889aac7b90e57a8f86cde6ffdd14724a4deac1a0881f41af5d391346: Status 404 returned error can't find the container with id e24d870b889aac7b90e57a8f86cde6ffdd14724a4deac1a0881f41af5d391346 Jul 14 17:59:57.559583 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.559565 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:59:57.811748 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.811718 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" event={"ID":"5f1dac4a-ae12-46d7-a951-3d89dd37f481","Type":"ContainerStarted","Data":"e24d870b889aac7b90e57a8f86cde6ffdd14724a4deac1a0881f41af5d391346"} Jul 14 17:59:57.813473 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:57.813454 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qszdz" event={"ID":"bd6d0995-b613-4d4d-9078-95a2d388fcc4","Type":"ContainerStarted","Data":"62957967a8043dba349785afa7cc5791d4ae9ed623556249a5dbcf40a9f4be2d"} Jul 14 17:59:58.818032 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:58.817994 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" event={"ID":"5f1dac4a-ae12-46d7-a951-3d89dd37f481","Type":"ContainerStarted","Data":"d4621cff8d3ad577dfba54834fd014bbb9c7b5276265ba2f738c51bfb3996d18"} Jul 14 17:59:58.818450 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:58.818068 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" event={"ID":"5f1dac4a-ae12-46d7-a951-3d89dd37f481","Type":"ContainerStarted","Data":"60f03e4e59f1f6b05a7a59a1402ec83df2458b72e8ff6471d48b5f29c7b616a1"} Jul 14 17:59:58.845717 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:58.845493 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-hxmn6" podStartSLOduration=0.794486545 podStartE2EDuration="1.845476681s" totalImagesPullingTime="1.050990136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:59:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:58.844993361 +0000 UTC m=+341.707783166" watchObservedRunningTime="2026-07-14 17:59:58.845476681 +0000 UTC m=+341.708266485" Jul 14 17:59:58.898657 ip-10-0-129-115 kubenswrapper[2572]: I0714 17:59:58.898555 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:00.283008 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.282977 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:00.510933 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.510900 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:00.544557 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.544477 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf"] Jul 14 18:00:00.548052 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.548029 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.550412 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.550390 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 14 18:00:00.550521 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.550388 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 14 18:00:00.550521 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.550450 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-g95fg\"" Jul 14 18:00:00.566990 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.566963 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-xdzxs"] Jul 14 18:00:00.570376 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.570343 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf"] Jul 14 18:00:00.570495 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.570480 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.573849 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.573821 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 14 18:00:00.574627 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.574593 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 14 18:00:00.575739 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.575716 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 14 18:00:00.576121 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.576104 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8knbk\"" Jul 14 18:00:00.583369 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.583343 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-xdzxs"] Jul 14 18:00:00.595929 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.595905 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s7tb7"] Jul 14 18:00:00.599020 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.599006 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.601184 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.601161 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7kfx4\"" Jul 14 18:00:00.601264 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.601246 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 18:00:00.602135 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.602117 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 18:00:00.602135 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.602133 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 18:00:00.618705 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.618674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:00.627746 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.627846 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627753 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-tls\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.627846 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627772 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7dbd9b71-d0de-48ed-9902-d02298af18be-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.627846 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.627951 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627849 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.627951 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627888 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-textfile\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.627951 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627915 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ac6b76-e411-4c43-8719-76e731d6ddd5-metrics-client-ca\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.628048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627950 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-sys\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.628048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.627981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-wtmp\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.628048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628003 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.628048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628030 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628054 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628073 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-root\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628105 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2l6j\" (UniqueName: \"kubernetes.io/projected/62ac6b76-e411-4c43-8719-76e731d6ddd5-kube-api-access-k2l6j\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628119 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfpdd\" (UniqueName: \"kubernetes.io/projected/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-kube-api-access-cfpdd\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628148 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dbd9b71-d0de-48ed-9902-d02298af18be-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628170 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.628187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.628185 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wwkx\" (UniqueName: \"kubernetes.io/projected/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-api-access-6wwkx\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.710590 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.710559 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:00.728852 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.728821 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-root\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.728852 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.728854 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729027 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.728877 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l6j\" (UniqueName: \"kubernetes.io/projected/62ac6b76-e411-4c43-8719-76e731d6ddd5-kube-api-access-k2l6j\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729027 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.728929 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-root\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729027 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.728967 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfpdd\" (UniqueName: \"kubernetes.io/projected/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-kube-api-access-cfpdd\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.729027 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.728997 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dbd9b71-d0de-48ed-9902-d02298af18be-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729052 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wwkx\" (UniqueName: \"kubernetes.io/projected/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-api-access-6wwkx\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729083 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-tls\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:00:00.729132 2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7dbd9b71-d0de-48ed-9902-d02298af18be-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.729226 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:00:00.729194 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-tls podName:9b9dd58d-05c5-4636-8ada-2dfeb3569c9c nodeName:}" failed. No retries permitted until 2026-07-14 18:00:01.229176058 +0000 UTC m=+344.091965858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-wtvlf" (UID: "9b9dd58d-05c5-4636-8ada-2dfeb3569c9c") : secret "openshift-state-metrics-tls" not found Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729286 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-textfile\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729368 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ac6b76-e411-4c43-8719-76e731d6ddd5-metrics-client-ca\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-sys\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729418 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-wtmp\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729443 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729445 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729533 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.729565 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729566 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7dbd9b71-d0de-48ed-9902-d02298af18be-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.730113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729832 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-textfile\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.730113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729837 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dbd9b71-d0de-48ed-9902-d02298af18be-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.730113 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:00:00.729912 2572 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 14 18:00:00.730113 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:00:00.729966 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-tls podName:7dbd9b71-d0de-48ed-9902-d02298af18be nodeName:}" failed. No retries permitted until 2026-07-14 18:00:01.229949152 +0000 UTC m=+344.092738951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-xdzxs" (UID: "7dbd9b71-d0de-48ed-9902-d02298af18be") : secret "kube-state-metrics-tls" not found Jul 14 18:00:00.730113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.729917 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-sys\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.730113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.730029 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-wtmp\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.730418 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.730255 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.730418 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.730344 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.730521 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.730455 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ac6b76-e411-4c43-8719-76e731d6ddd5-metrics-client-ca\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.731942 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.731911 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.732287 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.732268 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62ac6b76-e411-4c43-8719-76e731d6ddd5-node-exporter-tls\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.732366 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.732290 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.732422 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.732392 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.740101 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.740078 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfpdd\" (UniqueName: \"kubernetes.io/projected/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-kube-api-access-cfpdd\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:00.740276 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.740255 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wwkx\" (UniqueName: \"kubernetes.io/projected/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-api-access-6wwkx\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:00.741526 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.741505 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2l6j\" (UniqueName: \"kubernetes.io/projected/62ac6b76-e411-4c43-8719-76e731d6ddd5-kube-api-access-k2l6j\") pod \"node-exporter-s7tb7\" (UID: \"62ac6b76-e411-4c43-8719-76e731d6ddd5\") " pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.907793 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:00.907762 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s7tb7" Jul 14 18:00:00.919721 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:00:00.919685 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ac6b76_e411_4c43_8719_76e731d6ddd5.slice/crio-f1f747f6a20a317378db45c03d38dd2d2ddbadf51334f65b7dd918180a7dc2e3 WatchSource:0}: Error finding container f1f747f6a20a317378db45c03d38dd2d2ddbadf51334f65b7dd918180a7dc2e3: Status 404 returned error can't find the container with id f1f747f6a20a317378db45c03d38dd2d2ddbadf51334f65b7dd918180a7dc2e3 Jul 14 18:00:01.233466 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.233389 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:01.233466 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.233431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:01.235953 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.235934 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dbd9b71-d0de-48ed-9902-d02298af18be-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xdzxs\" (UID: \"7dbd9b71-d0de-48ed-9902-d02298af18be\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:01.236060 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.235934 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b9dd58d-05c5-4636-8ada-2dfeb3569c9c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wtvlf\" (UID: \"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:01.456909 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.456881 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" Jul 14 18:00:01.479348 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.479314 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" Jul 14 18:00:01.589852 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.589788 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 18:00:01.598089 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.598065 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.600496 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600328 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 18:00:01.600496 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600336 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 18:00:01.600496 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600429 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 18:00:01.600895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600596 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jtk4d\"" Jul 14 18:00:01.600895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600713 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 18:00:01.600895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600727 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 18:00:01.600895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.600795 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 18:00:01.601151 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.601015 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 18:00:01.601151 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.601051 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 18:00:01.601282 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.601236 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 18:00:01.618939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.611696 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf"] Jul 14 18:00:01.619735 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:00:01.619690 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b9dd58d_05c5_4636_8ada_2dfeb3569c9c.slice/crio-b0dea36cd202af6cdef8d6e2c37d9e99c284ee7c6b1a002b16ca297965cfb582 WatchSource:0}: Error finding container b0dea36cd202af6cdef8d6e2c37d9e99c284ee7c6b1a002b16ca297965cfb582: Status 404 returned error can't find the container with id b0dea36cd202af6cdef8d6e2c37d9e99c284ee7c6b1a002b16ca297965cfb582 Jul 14 18:00:01.620896 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.620876 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 18:00:01.635332 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.635266 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-xdzxs"] Jul 14 18:00:01.636509 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636490 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/995afcba-5533-4270-9dbe-f3fdde9dd0f8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636602 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636602 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636559 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636746 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636642 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-config-volume\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636746 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636666 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/995afcba-5533-4270-9dbe-f3fdde9dd0f8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636746 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636693 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636746 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995afcba-5533-4270-9dbe-f3fdde9dd0f8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636754 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636786 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs9tk\" (UniqueName: \"kubernetes.io/projected/995afcba-5533-4270-9dbe-f3fdde9dd0f8-kube-api-access-vs9tk\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636813 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636837 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995afcba-5533-4270-9dbe-f3fdde9dd0f8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636885 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-web-config\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.636943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.636910 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/995afcba-5533-4270-9dbe-f3fdde9dd0f8-config-out\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.638249 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:00:01.638225 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbd9b71_d0de_48ed_9902_d02298af18be.slice/crio-30361750a144e4dd8cbf5694e0468f6315e2f6031158b9474cbbf02c13144361 WatchSource:0}: Error finding container 30361750a144e4dd8cbf5694e0468f6315e2f6031158b9474cbbf02c13144361: Status 404 returned error can't find the container with id 30361750a144e4dd8cbf5694e0468f6315e2f6031158b9474cbbf02c13144361 Jul 14 18:00:01.738241 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738217 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738334 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738283 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-config-volume\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738334 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/995afcba-5533-4270-9dbe-f3fdde9dd0f8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738443 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738331 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738443 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738368 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995afcba-5533-4270-9dbe-f3fdde9dd0f8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738644 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738597 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738787 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738764 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs9tk\" (UniqueName: \"kubernetes.io/projected/995afcba-5533-4270-9dbe-f3fdde9dd0f8-kube-api-access-vs9tk\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.738905 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.738890 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.739163 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739117 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995afcba-5533-4270-9dbe-f3fdde9dd0f8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.739245 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739090 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995afcba-5533-4270-9dbe-f3fdde9dd0f8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.739389 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739371 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-web-config\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.739499 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739485 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/995afcba-5533-4270-9dbe-f3fdde9dd0f8-config-out\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.739658 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739596 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/995afcba-5533-4270-9dbe-f3fdde9dd0f8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.739806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739791 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.740308 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.739683 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/995afcba-5533-4270-9dbe-f3fdde9dd0f8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.741083 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.741026 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995afcba-5533-4270-9dbe-f3fdde9dd0f8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.743888 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.743845 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.743973 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.743906 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/995afcba-5533-4270-9dbe-f3fdde9dd0f8-config-out\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.744488 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.744416 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-web-config\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.744696 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.744649 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.744816 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.744798 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.744992 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.744951 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-config-volume\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.745084 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.745062 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.745292 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.745270 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/995afcba-5533-4270-9dbe-f3fdde9dd0f8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.746274 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.746249 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/995afcba-5533-4270-9dbe-f3fdde9dd0f8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.752442 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.752421 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs9tk\" (UniqueName: \"kubernetes.io/projected/995afcba-5533-4270-9dbe-f3fdde9dd0f8-kube-api-access-vs9tk\") pod \"alertmanager-main-0\" (UID: \"995afcba-5533-4270-9dbe-f3fdde9dd0f8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:01.829521 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.829490 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7tb7" event={"ID":"62ac6b76-e411-4c43-8719-76e731d6ddd5","Type":"ContainerStarted","Data":"f1f747f6a20a317378db45c03d38dd2d2ddbadf51334f65b7dd918180a7dc2e3"} Jul 14 18:00:01.830724 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.830701 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" event={"ID":"7dbd9b71-d0de-48ed-9902-d02298af18be","Type":"ContainerStarted","Data":"30361750a144e4dd8cbf5694e0468f6315e2f6031158b9474cbbf02c13144361"} Jul 14 18:00:01.832547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.832521 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" event={"ID":"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c","Type":"ContainerStarted","Data":"8d79998b5bece7f923c683645f643e921720e80ee7b21b838fd2d85e7227ed28"} Jul 14 18:00:01.832641 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.832564 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" event={"ID":"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c","Type":"ContainerStarted","Data":"16cf4e77dddd6932074a5e21fd0d4b18f798305746fdd67aa6b06fe34239deef"} Jul 14 18:00:01.832641 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.832582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" event={"ID":"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c","Type":"ContainerStarted","Data":"b0dea36cd202af6cdef8d6e2c37d9e99c284ee7c6b1a002b16ca297965cfb582"} Jul 14 18:00:01.909465 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:01.909427 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 18:00:02.054088 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:02.054053 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 18:00:02.056887 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:00:02.056857 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995afcba_5533_4270_9dbe_f3fdde9dd0f8.slice/crio-1606bb2125ff9ae3ec7269f45f2da5b16d186208f6cffdf035767177fc6b10d3 WatchSource:0}: Error finding container 1606bb2125ff9ae3ec7269f45f2da5b16d186208f6cffdf035767177fc6b10d3: Status 404 returned error can't find the container with id 1606bb2125ff9ae3ec7269f45f2da5b16d186208f6cffdf035767177fc6b10d3 Jul 14 18:00:02.839023 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:02.838930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"1606bb2125ff9ae3ec7269f45f2da5b16d186208f6cffdf035767177fc6b10d3"} Jul 14 18:00:02.840901 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:02.840868 2572 generic.go:356] "Generic (PLEG): container finished" podID="62ac6b76-e411-4c43-8719-76e731d6ddd5" containerID="1ed2eeaa717704da11ce2dd406ef0aad087e993224b6e9881632a9298c0aeb82" exitCode=0 Jul 14 18:00:02.841009 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:02.840914 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7tb7" event={"ID":"62ac6b76-e411-4c43-8719-76e731d6ddd5","Type":"ContainerDied","Data":"1ed2eeaa717704da11ce2dd406ef0aad087e993224b6e9881632a9298c0aeb82"} Jul 14 18:00:03.018437 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.018363 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:03.231967 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.231932 2572 patch_prober.go:28] interesting pod/image-registry-6db59c796c-v9grz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 18:00:03.232123 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.231982 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 18:00:03.271579 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.271539 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:03.463872 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.463798 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:03.564041 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.564013 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:03.847930 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.847890 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" event={"ID":"7dbd9b71-d0de-48ed-9902-d02298af18be","Type":"ContainerStarted","Data":"27ae890e35f0b43c6b12b28f50e40d7ac7f010022a805016977a1a8c4620a1b9"} Jul 14 18:00:03.848383 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.847959 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" event={"ID":"7dbd9b71-d0de-48ed-9902-d02298af18be","Type":"ContainerStarted","Data":"2f69e44ee1346facf73b146decfc9648c089623f5751bbc02fd123a3dc93f44c"} Jul 14 18:00:03.848383 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.847988 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" event={"ID":"7dbd9b71-d0de-48ed-9902-d02298af18be","Type":"ContainerStarted","Data":"8fe9308f9b8dd86dfdae4bf86a88c90c5ac44e9c20194eeac3df9a1a3ab8a457"} Jul 14 18:00:03.850530 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.850502 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" event={"ID":"9b9dd58d-05c5-4636-8ada-2dfeb3569c9c","Type":"ContainerStarted","Data":"027194bbef0509607b9125b8927a2b78d32e827881f045dc4e585cbb02618727"} Jul 14 18:00:03.853167 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.853017 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7tb7" event={"ID":"62ac6b76-e411-4c43-8719-76e731d6ddd5","Type":"ContainerStarted","Data":"7a5d43148d58b9eaf227c7a67be7ea003d11d9a78cebca995d911e0ded974dab"} Jul 14 18:00:03.853167 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.853107 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7tb7" event={"ID":"62ac6b76-e411-4c43-8719-76e731d6ddd5","Type":"ContainerStarted","Data":"b9f2eb4406f0f44e1e87fb488506545cc9d2d2cde19d6d47df6bc3d7204b098a"} Jul 14 18:00:03.855756 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.855729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:03.875173 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.872381 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-xdzxs" podStartSLOduration=1.996240018 podStartE2EDuration="3.872362692s" totalImagesPullingTime="1.876122674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:03.871718395 +0000 UTC m=+346.734508198" watchObservedRunningTime="2026-07-14 18:00:03.872362692 +0000 UTC m=+346.735152497" Jul 14 18:00:03.900921 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.900875 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s7tb7" podStartSLOduration=2.229177066 podStartE2EDuration="3.90085991s" totalImagesPullingTime="1.587712316s" totalInitContainerRuntime="83.970528ms" isStatefulPod=true podCreationTimestamp="2026-07-14 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:03.898864479 +0000 UTC m=+346.761654285" watchObservedRunningTime="2026-07-14 18:00:03.90085991 +0000 UTC m=+346.763649714" Jul 14 18:00:03.932184 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.932126 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wtvlf" podStartSLOduration=2.602266277 podStartE2EDuration="3.932110992s" totalImagesPullingTime="1.329844715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:03.931120365 +0000 UTC m=+346.793910166" watchObservedRunningTime="2026-07-14 18:00:03.932110992 +0000 UTC m=+346.794900795" Jul 14 18:00:03.944895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:03.944859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:04.858315 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:04.858282 2572 generic.go:356] "Generic (PLEG): container finished" podID="995afcba-5533-4270-9dbe-f3fdde9dd0f8" containerID="15dda0e9a719300a4f9681f1d0a6c0063257bd55baac7069ff9b77d7f31d8bdb" exitCode=0 Jul 14 18:00:04.858773 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:04.858370 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerDied","Data":"15dda0e9a719300a4f9681f1d0a6c0063257bd55baac7069ff9b77d7f31d8bdb"} Jul 14 18:00:06.382862 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:06.382826 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:06.484396 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:06.484360 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:06.722444 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:06.722362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:07.873832 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:07.873807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"1533a3d690c426f20cd093b3fb4b36e6f58e185ca602da9e6b79f032571c95df"} Jul 14 18:00:07.874171 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:07.873884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"ac04a0082618c8276870de9205344fa35f3ea8155325d98f54f57d6961c384ed"} Jul 14 18:00:07.874171 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:07.873926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"b55cca633ed385e5e59bd7f27ba7fc3c169f108b043448cd9f70d3cd743008ed"} Jul 14 18:00:08.246975 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.246884 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" containerID="cri-o://a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489" gracePeriod=30 Jul 14 18:00:08.880750 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.880714 2572 generic.go:356] "Generic (PLEG): container finished" podID="b1038199-9cee-4258-93fa-73efb2edfce6" containerID="a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489" exitCode=0 Jul 14 18:00:08.881230 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.880774 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" event={"ID":"b1038199-9cee-4258-93fa-73efb2edfce6","Type":"ContainerDied","Data":"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489"} Jul 14 18:00:08.881230 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.880871 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" event={"ID":"b1038199-9cee-4258-93fa-73efb2edfce6","Type":"ContainerStarted","Data":"f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf"} Jul 14 18:00:08.881230 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.881109 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 18:00:08.884368 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.884337 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"5c6d43e8710fc8b96954aae99608639be631b72d2e01cadc1961c6e8607abea2"} Jul 14 18:00:08.884478 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:08.884402 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"34e197e203914da372b4f1ba469cd5574e7f56ceeb92aedfb39cdcef972ab6fc"} Jul 14 18:00:09.891226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:09.891194 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"995afcba-5533-4270-9dbe-f3fdde9dd0f8","Type":"ContainerStarted","Data":"8a2dc67e8f8c279df2ec382799735a8bd1666b15dd9444d32ef82035c4775e95"} Jul 14 18:00:09.923187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:09.923142 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.03018517 podStartE2EDuration="8.923121456s" totalImagesPullingTime="5.875343346s" totalInitContainerRuntime="17.59294ms" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:09.922101969 +0000 UTC m=+352.784891773" watchObservedRunningTime="2026-07-14 18:00:09.923121456 +0000 UTC m=+352.785911302" Jul 14 18:00:10.604131 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:10.604091 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:10.703605 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:10.703571 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:13.090396 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:13.090342 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:13.190804 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:13.190772 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:16.294806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:16.294778 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6db59c796c-v9grz"] Jul 14 18:00:16.398381 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:16.398338 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:16.504959 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:16.504922 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:18.896969 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:18.896931 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:18.997567 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:18.997538 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:25.842180 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:25.842135 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:25.943942 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:25.943903 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:26.299904 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:26.299875 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 18:00:31.602839 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:31.602797 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 18:00:31.605128 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:31.605106 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae5d25a0-c8fc-4856-89f8-e47ac9945fa1-metrics-certs\") pod \"network-metrics-daemon-kmzpv\" (UID: \"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1\") " pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 18:00:31.816416 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:31.816386 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2kndq\"" Jul 14 18:00:31.823600 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:31.823579 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmzpv" Jul 14 18:00:31.955866 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:31.955767 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kmzpv"] Jul 14 18:00:31.958380 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:00:31.958351 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5d25a0_c8fc_4856_89f8_e47ac9945fa1.slice/crio-803e9fff99ed554fa669932b1188328e99e9a5e459389f679c2b4a7c7dc51cb5 WatchSource:0}: Error finding container 803e9fff99ed554fa669932b1188328e99e9a5e459389f679c2b4a7c7dc51cb5: Status 404 returned error can't find the container with id 803e9fff99ed554fa669932b1188328e99e9a5e459389f679c2b4a7c7dc51cb5 Jul 14 18:00:31.978833 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:31.978809 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmzpv" event={"ID":"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1","Type":"ContainerStarted","Data":"803e9fff99ed554fa669932b1188328e99e9a5e459389f679c2b4a7c7dc51cb5"} Jul 14 18:00:33.872659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:33.872600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:33.939306 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:33.939269 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:33.987766 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:33.987736 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmzpv" event={"ID":"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1","Type":"ContainerStarted","Data":"34883a8ee000bd436c9ab657294eade3afe10dcb83fd191316a2236599bc3d1b"} Jul 14 18:00:33.987905 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:33.987788 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmzpv" event={"ID":"ae5d25a0-c8fc-4856-89f8-e47ac9945fa1","Type":"ContainerStarted","Data":"94f3178bba69c5c2d6526c4d60af6076c74fa2770dfa50339e991777f05d8d60"} Jul 14 18:00:34.006272 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:34.006230 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kmzpv" podStartSLOduration=374.965411009 podStartE2EDuration="6m16.006215242s" totalImagesPullingTime="1.040804233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:34.005043253 +0000 UTC m=+376.867833059" watchObservedRunningTime="2026-07-14 18:00:34.006215242 +0000 UTC m=+376.869005045" Jul 14 18:00:34.153899 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:34.153809 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:34.246695 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:34.246662 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:41.314630 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.314574 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" containerID="cri-o://f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf" gracePeriod=30 Jul 14 18:00:41.561059 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.561037 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 18:00:41.586554 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586472 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-image-registry-private-configuration\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586554 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586539 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-bound-sa-token\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586564 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1038199-9cee-4258-93fa-73efb2edfce6-ca-trust-extracted\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586601 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-trusted-ca\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586654 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586697 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-registry-certificates\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586754 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-installation-pull-secrets\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.586908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.586782 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcnt9\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-kube-api-access-kcnt9\") pod \"b1038199-9cee-4258-93fa-73efb2edfce6\" (UID: \"b1038199-9cee-4258-93fa-73efb2edfce6\") " Jul 14 18:00:41.587541 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.587481 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:00:41.590076 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.589121 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:00:41.590292 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.590237 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:00:41.590759 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.590714 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:00:41.591300 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.591267 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:00:41.591300 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.591273 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-kube-api-access-kcnt9" (OuterVolumeSpecName: "kube-api-access-kcnt9") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "kube-api-access-kcnt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:00:41.592364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.592339 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:00:41.598205 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.598180 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1038199-9cee-4258-93fa-73efb2edfce6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b1038199-9cee-4258-93fa-73efb2edfce6" (UID: "b1038199-9cee-4258-93fa-73efb2edfce6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:00:41.687915 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687880 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-registry-tls\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.687915 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687909 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-registry-certificates\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.687915 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687918 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-installation-pull-secrets\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.688114 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687927 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcnt9\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-kube-api-access-kcnt9\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.688114 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687939 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1038199-9cee-4258-93fa-73efb2edfce6-image-registry-private-configuration\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.688114 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687947 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1038199-9cee-4258-93fa-73efb2edfce6-bound-sa-token\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.688114 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687955 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1038199-9cee-4258-93fa-73efb2edfce6-ca-trust-extracted\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:41.688114 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:41.687963 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1038199-9cee-4258-93fa-73efb2edfce6-trusted-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:00:42.020868 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.020783 2572 generic.go:356] "Generic (PLEG): container finished" podID="b1038199-9cee-4258-93fa-73efb2edfce6" containerID="f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf" exitCode=0 Jul 14 18:00:42.020868 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.020829 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" Jul 14 18:00:42.021033 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.020870 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" event={"ID":"b1038199-9cee-4258-93fa-73efb2edfce6","Type":"ContainerDied","Data":"f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf"} Jul 14 18:00:42.021033 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.020918 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db59c796c-v9grz" event={"ID":"b1038199-9cee-4258-93fa-73efb2edfce6","Type":"ContainerDied","Data":"e441c59ca3a36fc5b286be288fb624b30fbcee23ba3b5db8fdc5f9d0c1b4ea43"} Jul 14 18:00:42.021033 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.020938 2572 scope.go:117] "RemoveContainer" containerID="f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf" Jul 14 18:00:42.029215 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.029199 2572 scope.go:117] "RemoveContainer" containerID="a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489" Jul 14 18:00:42.036604 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.036589 2572 scope.go:117] "RemoveContainer" containerID="f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf" Jul 14 18:00:42.036962 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:00:42.036942 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf\": container with ID starting with f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf not found: ID does not exist" containerID="f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf" Jul 14 18:00:42.037040 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.036975 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf"} err="failed to get container status \"f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf\": rpc error: code = NotFound desc = could not find container \"f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf\": container with ID starting with f6ef20bd5a4ac70a3693b09f577503e60536b366fba21fd5bc67191693e68dcf not found: ID does not exist" Jul 14 18:00:42.037040 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.036997 2572 scope.go:117] "RemoveContainer" containerID="a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489" Jul 14 18:00:42.037211 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:00:42.037195 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489\": container with ID starting with a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489 not found: ID does not exist" containerID="a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489" Jul 14 18:00:42.037250 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.037218 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489"} err="failed to get container status \"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489\": rpc error: code = NotFound desc = could not find container \"a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489\": container with ID starting with a07755f91500eb44aa852cf33ee4e8ac7bb8bf8d73aefc9533dee6a5cfcc4489 not found: ID does not exist" Jul 14 18:00:42.045976 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.045952 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6db59c796c-v9grz"] Jul 14 18:00:42.049771 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:42.049752 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6db59c796c-v9grz"] Jul 14 18:00:43.718362 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:43.718329 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" path="/var/lib/kubelet/pods/b1038199-9cee-4258-93fa-73efb2edfce6/volumes" Jul 14 18:00:57.886494 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:57.886457 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:57.994620 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:57.994564 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:00:58.062483 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:00:58.062445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:19.815186 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:19.815150 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:20.116978 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:20.116884 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:24.918501 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.918466 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-66897b9758-7blqq"] Jul 14 18:01:24.919437 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.919416 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" Jul 14 18:01:24.919556 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.919546 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" Jul 14 18:01:24.919765 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.919755 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" Jul 14 18:01:24.919857 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.919848 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" Jul 14 18:01:24.925717 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.925687 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" Jul 14 18:01:24.926219 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.926046 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1038199-9cee-4258-93fa-73efb2edfce6" containerName="registry" Jul 14 18:01:24.930322 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.930302 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:24.933699 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.933677 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 18:01:24.933807 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.933721 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 18:01:24.933807 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.933763 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 18:01:24.933807 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.933762 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g2bc8\"" Jul 14 18:01:24.933966 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.933680 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 18:01:24.934032 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.934010 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 18:01:24.938684 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.938660 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66897b9758-7blqq"] Jul 14 18:01:24.940786 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:24.940192 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 18:01:25.010808 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.010771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.010808 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.010813 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-federate-client-tls\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.011006 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.010894 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.011006 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.010927 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-telemeter-client-tls\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.011006 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.010943 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs59z\" (UniqueName: \"kubernetes.io/projected/58c00766-dc64-4549-83d3-e3364ef1e6b6-kube-api-access-hs59z\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.011006 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.010981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-secret-telemeter-client\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.011129 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.011025 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-metrics-client-ca\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.011129 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.011051 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-serving-certs-ca-bundle\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.111818 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.111769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-secret-telemeter-client\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.111991 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.111834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-metrics-client-ca\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.111991 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.111867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-serving-certs-ca-bundle\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.111991 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.111900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.111991 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.111924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-federate-client-tls\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.111991 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.111977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.112228 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.112010 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-telemeter-client-tls\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.112228 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.112030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs59z\" (UniqueName: \"kubernetes.io/projected/58c00766-dc64-4549-83d3-e3364ef1e6b6-kube-api-access-hs59z\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.112657 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.112602 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-metrics-client-ca\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.113044 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.113022 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.113471 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.113451 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c00766-dc64-4549-83d3-e3364ef1e6b6-serving-certs-ca-bundle\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.114537 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.114513 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.114708 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.114687 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-secret-telemeter-client\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.115048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.115031 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-telemeter-client-tls\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.115109 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.115089 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58c00766-dc64-4549-83d3-e3364ef1e6b6-federate-client-tls\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.123260 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.123237 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs59z\" (UniqueName: \"kubernetes.io/projected/58c00766-dc64-4549-83d3-e3364ef1e6b6-kube-api-access-hs59z\") pod \"telemeter-client-66897b9758-7blqq\" (UID: \"58c00766-dc64-4549-83d3-e3364ef1e6b6\") " pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.244160 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.244072 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" Jul 14 18:01:25.413052 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:25.413018 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66897b9758-7blqq"] Jul 14 18:01:25.415958 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:01:25.415935 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c00766_dc64_4549_83d3_e3364ef1e6b6.slice/crio-da55d465f66d79f4941634b5f8293b328b16c241c8fafdd880c1e78a456b2315 WatchSource:0}: Error finding container da55d465f66d79f4941634b5f8293b328b16c241c8fafdd880c1e78a456b2315: Status 404 returned error can't find the container with id da55d465f66d79f4941634b5f8293b328b16c241c8fafdd880c1e78a456b2315 Jul 14 18:01:26.195973 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:26.195938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" event={"ID":"58c00766-dc64-4549-83d3-e3364ef1e6b6","Type":"ContainerStarted","Data":"da55d465f66d79f4941634b5f8293b328b16c241c8fafdd880c1e78a456b2315"} Jul 14 18:01:31.217197 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:31.217161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" event={"ID":"58c00766-dc64-4549-83d3-e3364ef1e6b6","Type":"ContainerStarted","Data":"809939baf936eb42ea55b701987dadd24cb3b6af2093bc8c7bd28203cdab90ce"} Jul 14 18:01:31.217626 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:31.217255 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" event={"ID":"58c00766-dc64-4549-83d3-e3364ef1e6b6","Type":"ContainerStarted","Data":"878c8d6e1582af50590f2134ef9311a011c31bc69c20ea96a0d53d7bbf5712c3"} Jul 14 18:01:31.217626 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:31.217286 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" event={"ID":"58c00766-dc64-4549-83d3-e3364ef1e6b6","Type":"ContainerStarted","Data":"f8cadcc57c892072d773609d4cd756c56e1fd9111c320f1d9ce70b1c103266c1"} Jul 14 18:01:31.249592 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:31.249549 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-66897b9758-7blqq" podStartSLOduration=2.299728382 podStartE2EDuration="7.249533633s" totalImagesPullingTime="4.949805251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:01:31.247213747 +0000 UTC m=+434.110003551" watchObservedRunningTime="2026-07-14 18:01:31.249533633 +0000 UTC m=+434.112323437" Jul 14 18:01:32.036089 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:32.036050 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:35.105123 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:35.105089 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:35.204113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:35.204080 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:47.156402 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:47.156317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:47.256779 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:47.256736 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:53.140402 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:53.140363 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:53.238745 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:53.238713 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:53.306114 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:53.306082 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:56.265255 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:56.265215 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:01:56.372150 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:01:56.372111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:05.681526 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:05.681493 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:05.782410 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:05.782376 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:24.673508 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:24.673458 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:29.031254 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:29.031220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:29.126209 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:29.126174 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:47.670675 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.670636 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:47.764457 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.764430 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h"] Jul 14 18:02:47.767758 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.767738 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.769730 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.769703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:47.770519 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.770502 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 18:02:47.771325 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.771306 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 18:02:47.771419 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.771341 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 18:02:47.771419 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.771316 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 18:02:47.780479 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.780458 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h"] Jul 14 18:02:47.804807 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.804780 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms"] Jul 14 18:02:47.807976 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.807961 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.810744 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.810711 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 14 18:02:47.810866 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.810763 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 14 18:02:47.810866 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.810771 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 14 18:02:47.810866 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.810763 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 14 18:02:47.822758 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.822739 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms"] Jul 14 18:02:47.855155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.855116 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:02:47.888295 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.888249 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/78664c84-e16b-4267-9074-22adcd909b82-klusterlet-config\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.888456 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.888335 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/78664c84-e16b-4267-9074-22adcd909b82-kube-api-access-tjbpw\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.888456 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.888366 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78664c84-e16b-4267-9074-22adcd909b82-tmp\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.989446 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78664c84-e16b-4267-9074-22adcd909b82-tmp\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.989446 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989410 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cd6a164-493a-4a70-9928-234e3894cc9d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.989446 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-ca\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.989693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/78664c84-e16b-4267-9074-22adcd909b82-klusterlet-config\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.989693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.989693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.989693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989544 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc6pp\" (UniqueName: \"kubernetes.io/projected/5cd6a164-493a-4a70-9928-234e3894cc9d-kube-api-access-xc6pp\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.989693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989586 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-hub\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:47.989693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989635 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/78664c84-e16b-4267-9074-22adcd909b82-kube-api-access-tjbpw\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.989874 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.989765 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78664c84-e16b-4267-9074-22adcd909b82-tmp\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:47.992051 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:47.992035 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/78664c84-e16b-4267-9074-22adcd909b82-klusterlet-config\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:48.003703 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.003681 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/78664c84-e16b-4267-9074-22adcd909b82-kube-api-access-tjbpw\") pod \"klusterlet-addon-workmgr-845d47495b-9mc9h\" (UID: \"78664c84-e16b-4267-9074-22adcd909b82\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:48.077978 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.077938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:48.090093 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.090072 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-hub\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.090158 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.090120 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cd6a164-493a-4a70-9928-234e3894cc9d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.090158 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.090137 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-ca\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.090307 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.090284 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.090370 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.090347 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.090421 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.090385 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc6pp\" (UniqueName: \"kubernetes.io/projected/5cd6a164-493a-4a70-9928-234e3894cc9d-kube-api-access-xc6pp\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.091514 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.091488 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cd6a164-493a-4a70-9928-234e3894cc9d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.093662 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.093643 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-hub\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.093775 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.093724 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-ca\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.094026 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.094002 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.094092 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.094070 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5cd6a164-493a-4a70-9928-234e3894cc9d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.101483 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.101461 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc6pp\" (UniqueName: \"kubernetes.io/projected/5cd6a164-493a-4a70-9928-234e3894cc9d-kube-api-access-xc6pp\") pod \"cluster-proxy-proxy-agent-744f5556c6-945ms\" (UID: \"5cd6a164-493a-4a70-9928-234e3894cc9d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.127651 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.127596 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" Jul 14 18:02:48.214649 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.214594 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h"] Jul 14 18:02:48.218418 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:02:48.218299 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78664c84_e16b_4267_9074_22adcd909b82.slice/crio-3436ffc18d58c58509ac938b027322cff5ff16d8b0e6fffb13ee133c4673b5fa WatchSource:0}: Error finding container 3436ffc18d58c58509ac938b027322cff5ff16d8b0e6fffb13ee133c4673b5fa: Status 404 returned error can't find the container with id 3436ffc18d58c58509ac938b027322cff5ff16d8b0e6fffb13ee133c4673b5fa Jul 14 18:02:48.270935 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.270803 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms"] Jul 14 18:02:48.273386 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:02:48.273359 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cd6a164_493a_4a70_9928_234e3894cc9d.slice/crio-4cfdf3b787231af6950b38d71a5ce1f6230bcbbfe2c98d5bee3de0389f078184 WatchSource:0}: Error finding container 4cfdf3b787231af6950b38d71a5ce1f6230bcbbfe2c98d5bee3de0389f078184: Status 404 returned error can't find the container with id 4cfdf3b787231af6950b38d71a5ce1f6230bcbbfe2c98d5bee3de0389f078184 Jul 14 18:02:48.522939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.522847 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" event={"ID":"78664c84-e16b-4267-9074-22adcd909b82","Type":"ContainerStarted","Data":"3436ffc18d58c58509ac938b027322cff5ff16d8b0e6fffb13ee133c4673b5fa"} Jul 14 18:02:48.523944 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:48.523918 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" event={"ID":"5cd6a164-493a-4a70-9928-234e3894cc9d","Type":"ContainerStarted","Data":"4cfdf3b787231af6950b38d71a5ce1f6230bcbbfe2c98d5bee3de0389f078184"} Jul 14 18:02:52.544447 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:52.544402 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" event={"ID":"5cd6a164-493a-4a70-9928-234e3894cc9d","Type":"ContainerStarted","Data":"6d7725fdc9b203c4a5545a4b686bfdd1b395016811ada0fa28f4e04a14c87353"} Jul 14 18:02:52.544930 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:52.544457 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" event={"ID":"5cd6a164-493a-4a70-9928-234e3894cc9d","Type":"ContainerStarted","Data":"64d25dcdb26fec35aa9356c9f0cc20dd1034b212f7c1465381b27b74e6f8b759"} Jul 14 18:02:52.544930 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:52.544474 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" event={"ID":"5cd6a164-493a-4a70-9928-234e3894cc9d","Type":"ContainerStarted","Data":"a78f05d65c28ba6a4bd4eef884fd47ffc5f3c8a2a811cd6d8158a2c045f58585"} Jul 14 18:02:52.583169 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:52.583109 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744f5556c6-945ms" podStartSLOduration=2.146955013 podStartE2EDuration="5.583091506s" totalImagesPullingTime="3.436136493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:02:52.582992815 +0000 UTC m=+515.445782621" watchObservedRunningTime="2026-07-14 18:02:52.583091506 +0000 UTC m=+515.445881310" Jul 14 18:02:54.556057 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:54.556015 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" event={"ID":"78664c84-e16b-4267-9074-22adcd909b82","Type":"ContainerStarted","Data":"b020c91f13d0116fea43ff8d4d837b8b417cd77cbf97676b1aafcb4e1599ebb3"} Jul 14 18:02:54.556468 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:54.556083 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:54.557422 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:54.557403 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" Jul 14 18:02:54.576708 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:02:54.576659 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-845d47495b-9mc9h" podStartSLOduration=1.338933308 podStartE2EDuration="7.576643161s" totalImagesPullingTime="6.237709853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:02:54.575357223 +0000 UTC m=+517.438147028" watchObservedRunningTime="2026-07-14 18:02:54.576643161 +0000 UTC m=+517.439432964" Jul 14 18:03:00.773926 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:00.773880 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:01.355175 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:01.355141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:04.277136 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:04.277096 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:05.585642 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:05.585585 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:05.694868 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:05.694823 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:07.874968 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:07.874936 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:08.024111 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:08.024077 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:10.051312 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:10.051224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:10.548101 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:10.548069 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:11.968267 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:11.968233 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:12.087712 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:12.087678 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:18.334306 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:18.334271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:18.810838 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:18.810812 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:18.957560 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:18.957528 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:24.159427 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:24.159388 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:24.725158 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:24.725126 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:24.818059 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:24.818029 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:26.674147 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:26.674113 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:38.815245 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:38.815205 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:38.916262 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:38.916225 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:40.615550 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:40.615513 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:40.715672 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:40.715636 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:41.220971 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.220937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:41.511676 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.511588 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:41.619143 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.619100 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:41.677552 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.677517 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7"] Jul 14 18:03:41.679908 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.679888 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.687002 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:41.686974 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" podUID="07baa099-c4a9-4cec-8b71-fd04badfec54" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.687363 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:41.687344 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-vfmqw\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vfmqw\"" type="*v1.Secret" Jul 14 18:03:41.687652 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:41.687629 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 14 18:03:41.687888 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:41.687871 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 14 18:03:41.688047 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:41.688028 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 14 18:03:41.688151 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:41.688133 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Jul 14 18:03:41.709708 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.709676 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7"] Jul 14 18:03:41.742073 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.742043 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.742266 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.742094 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.742266 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.742122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07baa099-c4a9-4cec-8b71-fd04badfec54-tmp\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.742266 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.742160 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z82lz\" (UniqueName: \"kubernetes.io/projected/07baa099-c4a9-4cec-8b71-fd04badfec54-kube-api-access-z82lz\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.742266 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.742202 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07baa099-c4a9-4cec-8b71-fd04badfec54-manifests\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.833604 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.833559 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:41.842599 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.842571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.842743 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.842635 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07baa099-c4a9-4cec-8b71-fd04badfec54-tmp\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.842743 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.842660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z82lz\" (UniqueName: \"kubernetes.io/projected/07baa099-c4a9-4cec-8b71-fd04badfec54-kube-api-access-z82lz\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.842743 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.842702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07baa099-c4a9-4cec-8b71-fd04badfec54-manifests\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.842899 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.842774 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.843099 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.843082 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07baa099-c4a9-4cec-8b71-fd04badfec54-manifests\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.844945 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.844923 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07baa099-c4a9-4cec-8b71-fd04badfec54-tmp\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:41.938763 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:41.938725 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:42.041824 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.041790 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:42.234583 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.234487 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:42.472845 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.472815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:42.521650 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.521554 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 18:03:42.650035 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.650000 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:42.843683 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:42.843645 2572 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 14 18:03:42.843905 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:42.843761 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-webhook-cert podName:07baa099-c4a9-4cec-8b71-fd04badfec54 nodeName:}" failed. No retries permitted until 2026-07-14 18:03:43.343742293 +0000 UTC m=+566.206532099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-webhook-cert") pod "opendatahub-operator-controller-manager-67c464fdbc-ffbk7" (UID: "07baa099-c4a9-4cec-8b71-fd04badfec54") : failed to sync secret cache: timed out waiting for the condition Jul 14 18:03:42.843905 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:42.843660 2572 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 14 18:03:42.843905 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:42.843860 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-apiservice-cert podName:07baa099-c4a9-4cec-8b71-fd04badfec54 nodeName:}" failed. No retries permitted until 2026-07-14 18:03:43.343845425 +0000 UTC m=+566.206635207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-apiservice-cert") pod "opendatahub-operator-controller-manager-67c464fdbc-ffbk7" (UID: "07baa099-c4a9-4cec-8b71-fd04badfec54") : failed to sync secret cache: timed out waiting for the condition Jul 14 18:03:42.849806 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:42.849784 2572 projected.go:291] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 14 18:03:42.867848 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.867825 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 18:03:42.994356 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:42.994326 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 18:03:43.000593 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:43.000574 2572 projected.go:196] Error preparing data for projected volume kube-api-access-z82lz for pod opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7: failed to sync configmap cache: timed out waiting for the condition Jul 14 18:03:43.000669 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:03:43.000658 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07baa099-c4a9-4cec-8b71-fd04badfec54-kube-api-access-z82lz podName:07baa099-c4a9-4cec-8b71-fd04badfec54 nodeName:}" failed. No retries permitted until 2026-07-14 18:03:43.500641015 +0000 UTC m=+566.363430798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z82lz" (UniqueName: "kubernetes.io/projected/07baa099-c4a9-4cec-8b71-fd04badfec54-kube-api-access-z82lz") pod "opendatahub-operator-controller-manager-67c464fdbc-ffbk7" (UID: "07baa099-c4a9-4cec-8b71-fd04badfec54") : failed to sync configmap cache: timed out waiting for the condition Jul 14 18:03:43.048729 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.048698 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 18:03:43.240725 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.240636 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vfmqw\"" Jul 14 18:03:43.356464 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.356417 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.356654 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.356493 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.359091 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.359056 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.359206 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.359103 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07baa099-c4a9-4cec-8b71-fd04badfec54-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.558135 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.558099 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z82lz\" (UniqueName: \"kubernetes.io/projected/07baa099-c4a9-4cec-8b71-fd04badfec54-kube-api-access-z82lz\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.560604 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.560585 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z82lz\" (UniqueName: \"kubernetes.io/projected/07baa099-c4a9-4cec-8b71-fd04badfec54-kube-api-access-z82lz\") pod \"opendatahub-operator-controller-manager-67c464fdbc-ffbk7\" (UID: \"07baa099-c4a9-4cec-8b71-fd04badfec54\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.790384 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.790345 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:43.939739 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:43.939714 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7"] Jul 14 18:03:43.941861 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:03:43.941828 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07baa099_c4a9_4cec_8b71_fd04badfec54.slice/crio-9065038023f6c5b3c5408b2add79ffb9e0d532a9c97c6291a2f5e7b7a82b5a25 WatchSource:0}: Error finding container 9065038023f6c5b3c5408b2add79ffb9e0d532a9c97c6291a2f5e7b7a82b5a25: Status 404 returned error can't find the container with id 9065038023f6c5b3c5408b2add79ffb9e0d532a9c97c6291a2f5e7b7a82b5a25 Jul 14 18:03:44.750036 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:44.749994 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" event={"ID":"07baa099-c4a9-4cec-8b71-fd04badfec54","Type":"ContainerStarted","Data":"9065038023f6c5b3c5408b2add79ffb9e0d532a9c97c6291a2f5e7b7a82b5a25"} Jul 14 18:03:47.765207 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:47.765170 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" event={"ID":"07baa099-c4a9-4cec-8b71-fd04badfec54","Type":"ContainerStarted","Data":"636999edfe72a707b28c5b9cf1706f5b80ac2e49db4a394cd1cf5470755c6dca"} Jul 14 18:03:48.583490 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:48.583453 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:48.772861 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:48.772826 2572 generic.go:356] "Generic (PLEG): container finished" podID="07baa099-c4a9-4cec-8b71-fd04badfec54" containerID="636999edfe72a707b28c5b9cf1706f5b80ac2e49db4a394cd1cf5470755c6dca" exitCode=0 Jul 14 18:03:48.773225 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:48.772886 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" event={"ID":"07baa099-c4a9-4cec-8b71-fd04badfec54","Type":"ContainerDied","Data":"636999edfe72a707b28c5b9cf1706f5b80ac2e49db4a394cd1cf5470755c6dca"} Jul 14 18:03:49.597261 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:49.597223 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:49.701381 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:49.701345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:49.780003 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:49.779968 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" event={"ID":"07baa099-c4a9-4cec-8b71-fd04badfec54","Type":"ContainerStarted","Data":"b7b73170752479b87be6a2841f4e276ee2d301436395a36a524d9566774ff4a8"} Jul 14 18:03:49.780420 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:49.780096 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:03:53.402540 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:53.402509 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:03:53.496912 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:03:53.496873 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:00.786518 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:00.786487 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" Jul 14 18:04:00.817833 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:00.817775 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-ffbk7" podStartSLOduration=15.541462329 podStartE2EDuration="19.817759278s" totalImagesPullingTime="3.198004107s" totalInitContainerRuntime="1.078292842s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:03:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:03:49.920191872 +0000 UTC m=+572.782981676" watchObservedRunningTime="2026-07-14 18:04:00.817759278 +0000 UTC m=+583.680549155" Jul 14 18:04:03.744149 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:03.744113 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:03.936126 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:03.936090 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:05.254632 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:05.254587 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:05.399942 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:05.399886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:05.554273 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:05.554245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:05.666768 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:05.666732 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:06.225282 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:06.225252 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:06.416405 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:06.416362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:06.617123 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:06.617095 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:06.764089 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:06.764058 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:07.910561 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.910524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:07.911014 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.910899 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x94sb"] Jul 14 18:04:07.921867 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.921843 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:07.926895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.926868 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 18:04:07.927060 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.926965 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 18:04:07.927060 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.926971 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 18:04:07.927176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.927087 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 18:04:07.927176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.927161 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 18:04:07.927176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.927159 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 18:04:07.927307 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.927246 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fbktg\"" Jul 14 18:04:07.929345 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.929325 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x94sb"] Jul 14 18:04:07.992701 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:07.992659 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:08.066290 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066256 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-cacerts\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.066290 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-token\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.066512 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066316 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.066512 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066421 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-598fs\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-kube-api-access-598fs\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.066512 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066459 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.066512 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8a6e19-4dac-478f-8a8a-571c336b12a0-local-certs\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.066716 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.066568 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167432 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-cacerts\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167432 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167382 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-token\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167432 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167783 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167456 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-598fs\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-kube-api-access-598fs\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167783 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167783 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8a6e19-4dac-478f-8a8a-571c336b12a0-local-certs\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.167783 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.167565 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.168653 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.168597 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.169996 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.169973 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8a6e19-4dac-478f-8a8a-571c336b12a0-local-certs\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.170100 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.170016 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.170173 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.170152 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.170226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.170162 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-cacerts\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.178445 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.178423 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-598fs\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-kube-api-access-598fs\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.178543 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.178468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-token\") pod \"istiod-openshift-gateway-7574459656-x94sb\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.233517 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.233482 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:08.374519 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.374485 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x94sb"] Jul 14 18:04:08.377510 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:04:08.377485 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8a6e19_4dac_478f_8a8a_571c336b12a0.slice/crio-e39387cb1850a8b06ca7ad825917a16dd96412536cbb6c30b1fd3b3a34cc34cf WatchSource:0}: Error finding container e39387cb1850a8b06ca7ad825917a16dd96412536cbb6c30b1fd3b3a34cc34cf: Status 404 returned error can't find the container with id e39387cb1850a8b06ca7ad825917a16dd96412536cbb6c30b1fd3b3a34cc34cf Jul 14 18:04:08.860865 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:08.860829 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" event={"ID":"7a8a6e19-4dac-478f-8a8a-571c336b12a0","Type":"ContainerStarted","Data":"e39387cb1850a8b06ca7ad825917a16dd96412536cbb6c30b1fd3b3a34cc34cf"} Jul 14 18:04:09.244449 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:09.244366 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:10.734975 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:10.734935 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:04:10.735234 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:10.735010 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:04:10.872600 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:10.872570 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" event={"ID":"7a8a6e19-4dac-478f-8a8a-571c336b12a0","Type":"ContainerStarted","Data":"ae93047d0a4b22d9b8d3a7ed214b30f31ea04ec451b6c32845c9154b3fb7ee93"} Jul 14 18:04:10.872776 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:10.872734 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:10.902464 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:10.902399 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" podStartSLOduration=1.547137499 podStartE2EDuration="3.902377452s" totalImagesPullingTime="2.355239953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:10.901080075 +0000 UTC m=+593.763869880" watchObservedRunningTime="2026-07-14 18:04:10.902377452 +0000 UTC m=+593.765167257" Jul 14 18:04:11.504164 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:11.504118 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:11.623454 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:11.623419 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:11.878505 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:11.878472 2572 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-x94sb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 18:04:11.878958 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:11.878527 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" podUID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 18:04:12.694166 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:12.694129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:12.901008 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:12.900974 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:13.032223 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:13.032129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:14.878194 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:14.878161 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:15.878390 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.878350 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4"] Jul 14 18:04:15.881832 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.881811 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:15.888026 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.888002 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 18:04:15.888894 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.888857 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 18:04:15.888955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.888887 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 18:04:15.889292 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.889272 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 18:04:15.889741 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.889725 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k6glj\"" Jul 14 18:04:15.895863 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.895835 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 18:04:15.927309 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:15.927281 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4"] Jul 14 18:04:16.045102 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.045070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3984a61e-a1ec-4916-ac8a-c2afc15376da-metrics-cert\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.045275 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.045116 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3984a61e-a1ec-4916-ac8a-c2afc15376da-cert\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.045275 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.045198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3984a61e-a1ec-4916-ac8a-c2afc15376da-manager-config\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.045275 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.045244 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs6w4\" (UniqueName: \"kubernetes.io/projected/3984a61e-a1ec-4916-ac8a-c2afc15376da-kube-api-access-zs6w4\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.146548 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.146453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3984a61e-a1ec-4916-ac8a-c2afc15376da-metrics-cert\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.146548 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.146511 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3984a61e-a1ec-4916-ac8a-c2afc15376da-cert\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.146548 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.146545 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3984a61e-a1ec-4916-ac8a-c2afc15376da-manager-config\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.146869 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.146567 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs6w4\" (UniqueName: \"kubernetes.io/projected/3984a61e-a1ec-4916-ac8a-c2afc15376da-kube-api-access-zs6w4\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.147341 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.147264 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3984a61e-a1ec-4916-ac8a-c2afc15376da-manager-config\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.149127 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.149101 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3984a61e-a1ec-4916-ac8a-c2afc15376da-metrics-cert\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.149239 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.149138 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3984a61e-a1ec-4916-ac8a-c2afc15376da-cert\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.157819 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.157798 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs6w4\" (UniqueName: \"kubernetes.io/projected/3984a61e-a1ec-4916-ac8a-c2afc15376da-kube-api-access-zs6w4\") pod \"lws-controller-manager-57cf6fdf7d-kcws4\" (UID: \"3984a61e-a1ec-4916-ac8a-c2afc15376da\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.191822 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.191784 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:16.331592 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.331548 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4"] Jul 14 18:04:16.343299 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:04:16.343260 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3984a61e_a1ec_4916_ac8a_c2afc15376da.slice/crio-d4eed20ae4ba52a952fc9506bc79ae7963b35d3e9b7085dd47dc1289494a113d WatchSource:0}: Error finding container d4eed20ae4ba52a952fc9506bc79ae7963b35d3e9b7085dd47dc1289494a113d: Status 404 returned error can't find the container with id d4eed20ae4ba52a952fc9506bc79ae7963b35d3e9b7085dd47dc1289494a113d Jul 14 18:04:16.899087 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:16.899046 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" event={"ID":"3984a61e-a1ec-4916-ac8a-c2afc15376da","Type":"ContainerStarted","Data":"d4eed20ae4ba52a952fc9506bc79ae7963b35d3e9b7085dd47dc1289494a113d"} Jul 14 18:04:17.762798 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:17.762719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:17.938590 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:17.938559 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:18.196388 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:18.196345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:19.762815 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.762782 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x94sb"] Jul 14 18:04:19.763176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.762970 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" podUID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" containerName="discovery" containerID="cri-o://ae93047d0a4b22d9b8d3a7ed214b30f31ea04ec451b6c32845c9154b3fb7ee93" gracePeriod=30 Jul 14 18:04:19.916248 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.916217 2572 generic.go:356] "Generic (PLEG): container finished" podID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" containerID="ae93047d0a4b22d9b8d3a7ed214b30f31ea04ec451b6c32845c9154b3fb7ee93" exitCode=0 Jul 14 18:04:19.916418 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.916286 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" event={"ID":"7a8a6e19-4dac-478f-8a8a-571c336b12a0","Type":"ContainerDied","Data":"ae93047d0a4b22d9b8d3a7ed214b30f31ea04ec451b6c32845c9154b3fb7ee93"} Jul 14 18:04:19.918427 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.918376 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" event={"ID":"3984a61e-a1ec-4916-ac8a-c2afc15376da","Type":"ContainerStarted","Data":"75ba9df1e1df1ed5832edc943aed44e87ef61af8cbb4972158fe688309a2370a"} Jul 14 18:04:19.918875 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.918843 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:19.965683 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:19.965634 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" podStartSLOduration=2.43581531 podStartE2EDuration="4.965591559s" totalImagesPullingTime="2.529776249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:19.963173305 +0000 UTC m=+602.825963110" watchObservedRunningTime="2026-07-14 18:04:19.965591559 +0000 UTC m=+602.828381363" Jul 14 18:04:20.013584 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.013525 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:20.182686 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182649 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-cacerts\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.182882 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182715 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-kubeconfig\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.182882 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182750 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-598fs\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-kube-api-access-598fs\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.182882 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182772 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-token\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.182882 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182800 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8a6e19-4dac-478f-8a8a-571c336b12a0-local-certs\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.182882 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182814 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-dns-cert\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.182882 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.182866 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-ca-configmap\") pod \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\" (UID: \"7a8a6e19-4dac-478f-8a8a-571c336b12a0\") " Jul 14 18:04:20.183317 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.183278 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:04:20.185390 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.185350 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:04:20.185489 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.185384 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8a6e19-4dac-478f-8a8a-571c336b12a0-local-certs" (OuterVolumeSpecName: "local-certs") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:04:20.185552 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.185506 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-cacerts" (OuterVolumeSpecName: "cacerts") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:04:20.185638 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.185604 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:04:20.185690 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.185636 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-kube-api-access-598fs" (OuterVolumeSpecName: "kube-api-access-598fs") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "kube-api-access-598fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:04:20.185729 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.185691 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7a8a6e19-4dac-478f-8a8a-571c336b12a0" (UID: "7a8a6e19-4dac-478f-8a8a-571c336b12a0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285726 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-token\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285773 2572 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8a6e19-4dac-478f-8a8a-571c336b12a0-local-certs\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285783 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-dns-cert\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285792 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-csr-ca-configmap\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285802 2572 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-cacerts\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285810 2572 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a8a6e19-4dac-478f-8a8a-571c336b12a0-istio-kubeconfig\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.285825 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.285822 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-598fs\" (UniqueName: \"kubernetes.io/projected/7a8a6e19-4dac-478f-8a8a-571c336b12a0-kube-api-access-598fs\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:04:20.777331 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.777298 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:20.924544 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.924510 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" Jul 14 18:04:20.924544 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.924537 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x94sb" event={"ID":"7a8a6e19-4dac-478f-8a8a-571c336b12a0","Type":"ContainerDied","Data":"e39387cb1850a8b06ca7ad825917a16dd96412536cbb6c30b1fd3b3a34cc34cf"} Jul 14 18:04:20.924775 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.924595 2572 scope.go:117] "RemoveContainer" containerID="ae93047d0a4b22d9b8d3a7ed214b30f31ea04ec451b6c32845c9154b3fb7ee93" Jul 14 18:04:20.971768 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.971735 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x94sb"] Jul 14 18:04:20.974161 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:20.974139 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x94sb"] Jul 14 18:04:21.077659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:21.077631 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:21.718408 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:21.718372 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" path="/var/lib/kubelet/pods/7a8a6e19-4dac-478f-8a8a-571c336b12a0/volumes" Jul 14 18:04:23.053676 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:23.053644 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:23.168683 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:23.168645 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:30.927842 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:30.927808 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-kcws4" Jul 14 18:04:52.073481 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:52.073392 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:04:55.239057 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:04:55.239021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:05.333834 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.333797 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:05.436725 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.436689 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:05.502220 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.502187 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7kf"] Jul 14 18:05:05.502541 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.502527 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" containerName="discovery" Jul 14 18:05:05.502585 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.502543 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" containerName="discovery" Jul 14 18:05:05.502637 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.502594 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8a6e19-4dac-478f-8a8a-571c336b12a0" containerName="discovery" Jul 14 18:05:05.509149 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.509125 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:05.512553 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.512527 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 18:05:05.512735 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.512582 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mhm8h\"" Jul 14 18:05:05.514504 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.514459 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 18:05:05.518483 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.518463 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7kf"] Jul 14 18:05:05.569663 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.569630 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:05.674840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.674747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm9z8\" (UniqueName: \"kubernetes.io/projected/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8-kube-api-access-sm9z8\") pod \"kuadrant-operator-catalog-rk7kf\" (UID: \"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8\") " pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:05.775346 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.775311 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm9z8\" (UniqueName: \"kubernetes.io/projected/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8-kube-api-access-sm9z8\") pod \"kuadrant-operator-catalog-rk7kf\" (UID: \"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8\") " pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:05.785917 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.785888 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm9z8\" (UniqueName: \"kubernetes.io/projected/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8-kube-api-access-sm9z8\") pod \"kuadrant-operator-catalog-rk7kf\" (UID: \"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8\") " pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:05.820391 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.820359 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:05.957214 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.957189 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7kf"] Jul 14 18:05:05.958986 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:05:05.958952 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8597f9bb_7827_4bbd_bf1e_6ffe951b1cc8.slice/crio-0ded2792c637dfe63d234509cd63e4eb30db2ff02beb966af013b83925b84b87 WatchSource:0}: Error finding container 0ded2792c637dfe63d234509cd63e4eb30db2ff02beb966af013b83925b84b87: Status 404 returned error can't find the container with id 0ded2792c637dfe63d234509cd63e4eb30db2ff02beb966af013b83925b84b87 Jul 14 18:05:05.960635 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:05.960588 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 18:05:06.072202 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.072167 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7kf"] Jul 14 18:05:06.094494 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.094464 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl6l2"] Jul 14 18:05:06.097618 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.097592 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:06.110131 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.110101 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl6l2"] Jul 14 18:05:06.123359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.123334 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" event={"ID":"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8","Type":"ContainerStarted","Data":"0ded2792c637dfe63d234509cd63e4eb30db2ff02beb966af013b83925b84b87"} Jul 14 18:05:06.279652 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.279536 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k82m\" (UniqueName: \"kubernetes.io/projected/4fb65699-86b8-4ed2-a549-dca404c03a49-kube-api-access-9k82m\") pod \"kuadrant-operator-catalog-xl6l2\" (UID: \"4fb65699-86b8-4ed2-a549-dca404c03a49\") " pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:06.284324 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.284296 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:06.380930 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.380893 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k82m\" (UniqueName: \"kubernetes.io/projected/4fb65699-86b8-4ed2-a549-dca404c03a49-kube-api-access-9k82m\") pod \"kuadrant-operator-catalog-xl6l2\" (UID: \"4fb65699-86b8-4ed2-a549-dca404c03a49\") " pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:06.391840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.391814 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k82m\" (UniqueName: \"kubernetes.io/projected/4fb65699-86b8-4ed2-a549-dca404c03a49-kube-api-access-9k82m\") pod \"kuadrant-operator-catalog-xl6l2\" (UID: \"4fb65699-86b8-4ed2-a549-dca404c03a49\") " pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:06.406603 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.406577 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:06.501275 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.501236 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:06.545907 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:06.545757 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl6l2"] Jul 14 18:05:06.548502 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:05:06.548472 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb65699_86b8_4ed2_a549_dca404c03a49.slice/crio-3ff60d6489c5fa09f88cccdbcefe47a8bf803a09933f3449cddb8e496cdeea57 WatchSource:0}: Error finding container 3ff60d6489c5fa09f88cccdbcefe47a8bf803a09933f3449cddb8e496cdeea57: Status 404 returned error can't find the container with id 3ff60d6489c5fa09f88cccdbcefe47a8bf803a09933f3449cddb8e496cdeea57 Jul 14 18:05:07.130625 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:07.130579 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" event={"ID":"4fb65699-86b8-4ed2-a549-dca404c03a49","Type":"ContainerStarted","Data":"3ff60d6489c5fa09f88cccdbcefe47a8bf803a09933f3449cddb8e496cdeea57"} Jul 14 18:05:09.142071 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.142035 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" event={"ID":"4fb65699-86b8-4ed2-a549-dca404c03a49","Type":"ContainerStarted","Data":"47209a9d347e014abcd2226f08ecfd8a39ee36a211edff7027597ec7caaad103"} Jul 14 18:05:09.143402 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.143378 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" event={"ID":"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8","Type":"ContainerStarted","Data":"ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022"} Jul 14 18:05:09.143523 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.143459 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" podUID="8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" containerName="registry-server" containerID="cri-o://ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022" gracePeriod=2 Jul 14 18:05:09.165262 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.165222 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" podStartSLOduration=1.47669016 podStartE2EDuration="3.165207247s" totalImagesPullingTime="1.688517087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:09.163265699 +0000 UTC m=+652.026055503" watchObservedRunningTime="2026-07-14 18:05:09.165207247 +0000 UTC m=+652.027997028" Jul 14 18:05:09.383577 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.383552 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:09.410724 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.410663 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm9z8\" (UniqueName: \"kubernetes.io/projected/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8-kube-api-access-sm9z8\") pod \"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8\" (UID: \"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8\") " Jul 14 18:05:09.412932 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.412902 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8-kube-api-access-sm9z8" (OuterVolumeSpecName: "kube-api-access-sm9z8") pod "8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" (UID: "8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8"). InnerVolumeSpecName "kube-api-access-sm9z8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:05:09.511450 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:09.511405 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm9z8\" (UniqueName: \"kubernetes.io/projected/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8-kube-api-access-sm9z8\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:05:10.148841 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.148812 2572 generic.go:356] "Generic (PLEG): container finished" podID="8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" containerID="ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022" exitCode=0 Jul 14 18:05:10.149289 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.148854 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" Jul 14 18:05:10.149289 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.148887 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" event={"ID":"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8","Type":"ContainerDied","Data":"ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022"} Jul 14 18:05:10.149289 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.148926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7kf" event={"ID":"8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8","Type":"ContainerDied","Data":"0ded2792c637dfe63d234509cd63e4eb30db2ff02beb966af013b83925b84b87"} Jul 14 18:05:10.149289 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.148946 2572 scope.go:117] "RemoveContainer" containerID="ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022" Jul 14 18:05:10.157650 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.157630 2572 scope.go:117] "RemoveContainer" containerID="ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022" Jul 14 18:05:10.157920 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:05:10.157897 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022\": container with ID starting with ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022 not found: ID does not exist" containerID="ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022" Jul 14 18:05:10.157990 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.157930 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022"} err="failed to get container status \"ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022\": rpc error: code = NotFound desc = could not find container \"ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022\": container with ID starting with ff49bcbd3cc02ad4265cf1b3310e62d0ff37fd50dd36deffc47821802975c022 not found: ID does not exist" Jul 14 18:05:10.170422 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.170398 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7kf"] Jul 14 18:05:10.174517 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:10.174488 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7kf"] Jul 14 18:05:11.719316 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:11.719279 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" path="/var/lib/kubelet/pods/8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8/volumes" Jul 14 18:05:16.407716 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:16.407674 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:16.408196 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:16.407746 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:16.430419 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:16.430389 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:17.208786 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:17.208759 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xl6l2" Jul 14 18:05:26.496364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:26.496328 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:26.577369 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:26.577334 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:28.920149 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:28.920115 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:29.531282 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:29.531247 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:30.132910 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:30.132877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:30.242752 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:30.242716 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:32.835113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.835073 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-c92gl"] Jul 14 18:05:32.835588 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.835567 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" containerName="registry-server" Jul 14 18:05:32.835687 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.835591 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" containerName="registry-server" Jul 14 18:05:32.835748 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.835722 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8597f9bb-7827-4bbd-bf1e-6ffe951b1cc8" containerName="registry-server" Jul 14 18:05:32.838242 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.838215 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:32.841739 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.841722 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wtwfn\"" Jul 14 18:05:32.864844 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.864815 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-c92gl"] Jul 14 18:05:32.909457 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:32.909423 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx7nn\" (UniqueName: \"kubernetes.io/projected/8c0bc7c6-9e4e-4244-a856-1229980c8c73-kube-api-access-nx7nn\") pod \"authorino-operator-75cc8d5cb7-c92gl\" (UID: \"8c0bc7c6-9e4e-4244-a856-1229980c8c73\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:33.010635 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:33.010567 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx7nn\" (UniqueName: \"kubernetes.io/projected/8c0bc7c6-9e4e-4244-a856-1229980c8c73-kube-api-access-nx7nn\") pod \"authorino-operator-75cc8d5cb7-c92gl\" (UID: \"8c0bc7c6-9e4e-4244-a856-1229980c8c73\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:33.030334 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:33.030301 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx7nn\" (UniqueName: \"kubernetes.io/projected/8c0bc7c6-9e4e-4244-a856-1229980c8c73-kube-api-access-nx7nn\") pod \"authorino-operator-75cc8d5cb7-c92gl\" (UID: \"8c0bc7c6-9e4e-4244-a856-1229980c8c73\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:33.147664 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:33.147542 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:33.289182 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:33.289148 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-c92gl"] Jul 14 18:05:33.292558 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:05:33.292530 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0bc7c6_9e4e_4244_a856_1229980c8c73.slice/crio-69b4ee0a5a3690c2927aa262fa0ec8d5f4aab3155e39170d79cbbd56c0c65e90 WatchSource:0}: Error finding container 69b4ee0a5a3690c2927aa262fa0ec8d5f4aab3155e39170d79cbbd56c0c65e90: Status 404 returned error can't find the container with id 69b4ee0a5a3690c2927aa262fa0ec8d5f4aab3155e39170d79cbbd56c0c65e90 Jul 14 18:05:33.955990 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:33.955954 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:34.259174 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:34.259084 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" event={"ID":"8c0bc7c6-9e4e-4244-a856-1229980c8c73","Type":"ContainerStarted","Data":"69b4ee0a5a3690c2927aa262fa0ec8d5f4aab3155e39170d79cbbd56c0c65e90"} Jul 14 18:05:35.560664 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.560628 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h"] Jul 14 18:05:35.562955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.562938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:35.569696 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.569379 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zmwn6\"" Jul 14 18:05:35.569696 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.569676 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 14 18:05:35.597219 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.597189 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h"] Jul 14 18:05:35.634839 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.634795 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4sq\" (UniqueName: \"kubernetes.io/projected/78fc3549-d46c-4343-9bbd-d27f67c53fc6-kube-api-access-jw4sq\") pod \"dns-operator-controller-manager-864dd78c4c-2485h\" (UID: \"78fc3549-d46c-4343-9bbd-d27f67c53fc6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:35.735532 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.735492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4sq\" (UniqueName: \"kubernetes.io/projected/78fc3549-d46c-4343-9bbd-d27f67c53fc6-kube-api-access-jw4sq\") pod \"dns-operator-controller-manager-864dd78c4c-2485h\" (UID: \"78fc3549-d46c-4343-9bbd-d27f67c53fc6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:35.791085 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.791056 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4sq\" (UniqueName: \"kubernetes.io/projected/78fc3549-d46c-4343-9bbd-d27f67c53fc6-kube-api-access-jw4sq\") pod \"dns-operator-controller-manager-864dd78c4c-2485h\" (UID: \"78fc3549-d46c-4343-9bbd-d27f67c53fc6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:35.873837 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:35.873745 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:36.108973 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:36.108946 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h"] Jul 14 18:05:36.110396 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:05:36.110363 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fc3549_d46c_4343_9bbd_d27f67c53fc6.slice/crio-c5d868599f2713a2b9293f3733eb799abd1adc91625dfa2d225beaed05db785d WatchSource:0}: Error finding container c5d868599f2713a2b9293f3733eb799abd1adc91625dfa2d225beaed05db785d: Status 404 returned error can't find the container with id c5d868599f2713a2b9293f3733eb799abd1adc91625dfa2d225beaed05db785d Jul 14 18:05:36.271517 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:36.271430 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" event={"ID":"78fc3549-d46c-4343-9bbd-d27f67c53fc6","Type":"ContainerStarted","Data":"c5d868599f2713a2b9293f3733eb799abd1adc91625dfa2d225beaed05db785d"} Jul 14 18:05:37.013845 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:37.013817 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:37.280969 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:37.280865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" event={"ID":"8c0bc7c6-9e4e-4244-a856-1229980c8c73","Type":"ContainerStarted","Data":"1855f74d88250bc1b01eed077fb83184daf16ee2e554fbd86665ff6117232cc1"} Jul 14 18:05:37.281144 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:37.281007 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:37.669851 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:37.669816 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:38.833476 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:38.833441 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:40.295667 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:40.295546 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" event={"ID":"78fc3549-d46c-4343-9bbd-d27f67c53fc6","Type":"ContainerStarted","Data":"414f8696d3d4d6a73f404aa2160a24a47746b3287fba2d8381ce430e1eeb11f5"} Jul 14 18:05:40.295667 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:40.295632 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:40.356229 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:40.356170 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" podStartSLOduration=5.214279081 podStartE2EDuration="8.356152726s" totalImagesPullingTime="3.141873645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:37.305678852 +0000 UTC m=+680.168468670" watchObservedRunningTime="2026-07-14 18:05:40.356152726 +0000 UTC m=+683.218942530" Jul 14 18:05:40.356895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:40.356868 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" podStartSLOduration=1.504256295 podStartE2EDuration="5.356859445s" totalImagesPullingTime="3.85260315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:40.355473581 +0000 UTC m=+683.218263398" watchObservedRunningTime="2026-07-14 18:05:40.356859445 +0000 UTC m=+683.219649313" Jul 14 18:05:41.961200 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:41.961167 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m"] Jul 14 18:05:41.980051 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:41.980026 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:41.984120 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:41.984097 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zbch6\"" Jul 14 18:05:41.989168 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:41.989145 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m"] Jul 14 18:05:42.092428 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.092391 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddwbb\" (UniqueName: \"kubernetes.io/projected/e3eef591-a715-461d-a124-288316332b36-kube-api-access-ddwbb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.092428 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.092440 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3eef591-a715-461d-a124-288316332b36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.193471 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.193431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3eef591-a715-461d-a124-288316332b36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.193681 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.193563 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddwbb\" (UniqueName: \"kubernetes.io/projected/e3eef591-a715-461d-a124-288316332b36-kube-api-access-ddwbb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.193876 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.193856 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3eef591-a715-461d-a124-288316332b36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.205522 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.205500 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddwbb\" (UniqueName: \"kubernetes.io/projected/e3eef591-a715-461d-a124-288316332b36-kube-api-access-ddwbb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.290945 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.290841 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:42.443247 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.443095 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m"] Jul 14 18:05:42.445973 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:05:42.445947 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3eef591_a715_461d_a124_288316332b36.slice/crio-9839c01b1f6f22fb722d7018e96a3cd4073113e17c1256c2148c8b6984237b39 WatchSource:0}: Error finding container 9839c01b1f6f22fb722d7018e96a3cd4073113e17c1256c2148c8b6984237b39: Status 404 returned error can't find the container with id 9839c01b1f6f22fb722d7018e96a3cd4073113e17c1256c2148c8b6984237b39 Jul 14 18:05:42.734693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:42.734657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:43.311558 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:43.311520 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" event={"ID":"e3eef591-a715-461d-a124-288316332b36","Type":"ContainerStarted","Data":"9839c01b1f6f22fb722d7018e96a3cd4073113e17c1256c2148c8b6984237b39"} Jul 14 18:05:46.965584 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:46.965547 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:48.236637 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:48.236544 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:48.288083 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:48.288052 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c92gl" Jul 14 18:05:48.341960 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:48.341915 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:50.341311 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:50.341270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:50.443661 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:50.443601 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:51.303069 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:51.303038 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2485h" Jul 14 18:05:51.358040 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:51.358007 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" event={"ID":"e3eef591-a715-461d-a124-288316332b36","Type":"ContainerStarted","Data":"8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a"} Jul 14 18:05:51.358525 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:51.358115 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:05:51.385410 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:51.385361 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" podStartSLOduration=1.806297451 podStartE2EDuration="10.385345921s" totalImagesPullingTime="8.57904847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:51.38529176 +0000 UTC m=+694.248081576" watchObservedRunningTime="2026-07-14 18:05:51.385345921 +0000 UTC m=+694.248135725" Jul 14 18:05:53.800939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.800903 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5"] Jul 14 18:05:53.803281 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.803265 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:53.805940 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.805916 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mh459\"" Jul 14 18:05:53.806828 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.806812 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 14 18:05:53.806927 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.806835 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 14 18:05:53.814060 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.814034 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:53.814447 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.814426 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5"] Jul 14 18:05:53.900211 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.900180 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea7ac2de-c1c5-47a7-baf8-d507ede55274-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:53.900379 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.900240 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea7ac2de-c1c5-47a7-baf8-d507ede55274-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:53.900379 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.900322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nz6j\" (UniqueName: \"kubernetes.io/projected/ea7ac2de-c1c5-47a7-baf8-d507ede55274-kube-api-access-7nz6j\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:53.926870 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:53.926833 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:54.001456 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.001425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nz6j\" (UniqueName: \"kubernetes.io/projected/ea7ac2de-c1c5-47a7-baf8-d507ede55274-kube-api-access-7nz6j\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.001689 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.001495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea7ac2de-c1c5-47a7-baf8-d507ede55274-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.001689 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.001535 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea7ac2de-c1c5-47a7-baf8-d507ede55274-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.002139 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.002113 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ea7ac2de-c1c5-47a7-baf8-d507ede55274-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.004066 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.004039 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea7ac2de-c1c5-47a7-baf8-d507ede55274-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.011241 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.011215 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nz6j\" (UniqueName: \"kubernetes.io/projected/ea7ac2de-c1c5-47a7-baf8-d507ede55274-kube-api-access-7nz6j\") pod \"kuadrant-console-plugin-66b5774985-8brd5\" (UID: \"ea7ac2de-c1c5-47a7-baf8-d507ede55274\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.112231 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.112142 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" Jul 14 18:05:54.251455 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.251369 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5"] Jul 14 18:05:54.253969 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:05:54.253941 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea7ac2de_c1c5_47a7_baf8_d507ede55274.slice/crio-ad3dbfd43314cd38e93cb621827137a6bc08afe3dfa26bd1949905a562e48f1f WatchSource:0}: Error finding container ad3dbfd43314cd38e93cb621827137a6bc08afe3dfa26bd1949905a562e48f1f: Status 404 returned error can't find the container with id ad3dbfd43314cd38e93cb621827137a6bc08afe3dfa26bd1949905a562e48f1f Jul 14 18:05:54.372409 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:54.372329 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" event={"ID":"ea7ac2de-c1c5-47a7-baf8-d507ede55274","Type":"ContainerStarted","Data":"ad3dbfd43314cd38e93cb621827137a6bc08afe3dfa26bd1949905a562e48f1f"} Jul 14 18:05:55.977716 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:55.977668 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:05:56.102931 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:05:56.102891 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:02.365282 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:02.365249 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:03.409855 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:03.409819 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:03.725113 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:03.725021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:04.658200 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.658164 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m"] Jul 14 18:06:04.658690 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.658404 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" podUID="e3eef591-a715-461d-a124-288316332b36" containerName="manager" containerID="cri-o://8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a" gracePeriod=2 Jul 14 18:06:04.662453 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.662427 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m"] Jul 14 18:06:04.744569 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.744541 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p"] Jul 14 18:06:04.744980 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.744964 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3eef591-a715-461d-a124-288316332b36" containerName="manager" Jul 14 18:06:04.744980 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.744982 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3eef591-a715-461d-a124-288316332b36" containerName="manager" Jul 14 18:06:04.745107 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.745083 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3eef591-a715-461d-a124-288316332b36" containerName="manager" Jul 14 18:06:04.753891 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.753869 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:04.756926 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:04.756897 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" podUID="e3eef591-a715-461d-a124-288316332b36" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:04.777684 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.777656 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p"] Jul 14 18:06:04.806949 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.806919 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926e1eab-5155-41b4-8305-e95ce7e93417-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xjs2p\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:04.807110 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.807078 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvzfm\" (UniqueName: \"kubernetes.io/projected/926e1eab-5155-41b4-8305-e95ce7e93417-kube-api-access-hvzfm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xjs2p\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:04.908163 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.908125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvzfm\" (UniqueName: \"kubernetes.io/projected/926e1eab-5155-41b4-8305-e95ce7e93417-kube-api-access-hvzfm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xjs2p\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:04.908409 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.908220 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926e1eab-5155-41b4-8305-e95ce7e93417-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xjs2p\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:04.908716 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.908692 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926e1eab-5155-41b4-8305-e95ce7e93417-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xjs2p\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:04.938968 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:04.938936 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvzfm\" (UniqueName: \"kubernetes.io/projected/926e1eab-5155-41b4-8305-e95ce7e93417-kube-api-access-hvzfm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xjs2p\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:05.066870 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:05.066834 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:06.068426 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:06.068391 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:06.221746 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:06.221706 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:07.045181 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:07.045152 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:07.176072 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:07.176037 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:08.646159 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:08.646069 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:08.787103 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:08.787066 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:13.863255 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:13.863168 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:13.968849 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:13.968815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:17.959180 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:17.959158 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:17.963634 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:17.963593 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" podUID="e3eef591-a715-461d-a124-288316332b36" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:17.975009 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:17.974983 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p"] Jul 14 18:06:17.976425 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:06:17.976401 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926e1eab_5155_41b4_8305_e95ce7e93417.slice/crio-cb32432bce23616a45f33a01947b76e25459dfe2915f6ac034d1a2b6dee8c633 WatchSource:0}: Error finding container cb32432bce23616a45f33a01947b76e25459dfe2915f6ac034d1a2b6dee8c633: Status 404 returned error can't find the container with id cb32432bce23616a45f33a01947b76e25459dfe2915f6ac034d1a2b6dee8c633 Jul 14 18:06:18.035685 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.035663 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3eef591-a715-461d-a124-288316332b36-extensions-socket-volume\") pod \"e3eef591-a715-461d-a124-288316332b36\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " Jul 14 18:06:18.035805 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.035703 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddwbb\" (UniqueName: \"kubernetes.io/projected/e3eef591-a715-461d-a124-288316332b36-kube-api-access-ddwbb\") pod \"e3eef591-a715-461d-a124-288316332b36\" (UID: \"e3eef591-a715-461d-a124-288316332b36\") " Jul 14 18:06:18.036180 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.036159 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3eef591-a715-461d-a124-288316332b36-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e3eef591-a715-461d-a124-288316332b36" (UID: "e3eef591-a715-461d-a124-288316332b36"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:06:18.037781 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.037761 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3eef591-a715-461d-a124-288316332b36-kube-api-access-ddwbb" (OuterVolumeSpecName: "kube-api-access-ddwbb") pod "e3eef591-a715-461d-a124-288316332b36" (UID: "e3eef591-a715-461d-a124-288316332b36"). InnerVolumeSpecName "kube-api-access-ddwbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:06:18.136314 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.136235 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3eef591-a715-461d-a124-288316332b36-extensions-socket-volume\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:06:18.136314 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.136260 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddwbb\" (UniqueName: \"kubernetes.io/projected/e3eef591-a715-461d-a124-288316332b36-kube-api-access-ddwbb\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:06:18.511836 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.511745 2572 generic.go:356] "Generic (PLEG): container finished" podID="e3eef591-a715-461d-a124-288316332b36" containerID="8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a" exitCode=0 Jul 14 18:06:18.511836 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.511805 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:18.512066 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.511843 2572 scope.go:117] "RemoveContainer" containerID="8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a" Jul 14 18:06:18.513747 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.513711 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" event={"ID":"926e1eab-5155-41b4-8305-e95ce7e93417","Type":"ContainerStarted","Data":"b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd"} Jul 14 18:06:18.513856 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.513797 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" event={"ID":"926e1eab-5155-41b4-8305-e95ce7e93417","Type":"ContainerStarted","Data":"cb32432bce23616a45f33a01947b76e25459dfe2915f6ac034d1a2b6dee8c633"} Jul 14 18:06:18.513856 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.513826 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:18.515275 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.515245 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" event={"ID":"ea7ac2de-c1c5-47a7-baf8-d507ede55274","Type":"ContainerStarted","Data":"09872848a7629b382129467399a0d731319ef911efa5bce62fb4490151dddc8e"} Jul 14 18:06:18.515467 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:18.515441 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" podUID="e3eef591-a715-461d-a124-288316332b36" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:18.521006 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.520992 2572 scope.go:117] "RemoveContainer" containerID="8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a" Jul 14 18:06:18.521270 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:18.521252 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a\": container with ID starting with 8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a not found: ID does not exist" containerID="8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a" Jul 14 18:06:18.521335 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.521277 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a"} err="failed to get container status \"8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a\": rpc error: code = NotFound desc = could not find container \"8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a\": container with ID starting with 8a14e357e65bb036dba054a63e6ca52dd32be23e4c9db95c67d7264ee799ef8a not found: ID does not exist" Jul 14 18:06:18.561396 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.561337 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8brd5" podStartSLOduration=1.93639337 podStartE2EDuration="25.56132167s" totalImagesPullingTime="23.6249283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:18.559761316 +0000 UTC m=+721.422551120" watchObservedRunningTime="2026-07-14 18:06:18.56132167 +0000 UTC m=+721.424111474" Jul 14 18:06:18.611302 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:18.611246 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" podStartSLOduration=14.611228009 podStartE2EDuration="14.611228009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:18.609318911 +0000 UTC m=+721.472108750" watchObservedRunningTime="2026-07-14 18:06:18.611228009 +0000 UTC m=+721.474017812" Jul 14 18:06:18.611471 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:18.611340 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" podUID="e3eef591-a715-461d-a124-288316332b36" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:18.613553 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:18.613533 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nbn2m\" is forbidden: User \"system:node:ip-10-0-129-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-115.ec2.internal' and this object" podUID="e3eef591-a715-461d-a124-288316332b36" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nbn2m" Jul 14 18:06:19.718686 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:19.718653 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3eef591-a715-461d-a124-288316332b36" path="/var/lib/kubelet/pods/e3eef591-a715-461d-a124-288316332b36/volumes" Jul 14 18:06:21.138817 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:21.138788 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:22.257172 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:22.257128 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:23.743689 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:23.743658 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p"] Jul 14 18:06:23.744091 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:23.743848 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" podUID="926e1eab-5155-41b4-8305-e95ce7e93417" containerName="manager" containerID="cri-o://b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd" gracePeriod=10 Jul 14 18:06:23.745566 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:23.745546 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:23.988851 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:23.988827 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:24.087109 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.087071 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926e1eab-5155-41b4-8305-e95ce7e93417-extensions-socket-volume\") pod \"926e1eab-5155-41b4-8305-e95ce7e93417\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " Jul 14 18:06:24.087276 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.087124 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvzfm\" (UniqueName: \"kubernetes.io/projected/926e1eab-5155-41b4-8305-e95ce7e93417-kube-api-access-hvzfm\") pod \"926e1eab-5155-41b4-8305-e95ce7e93417\" (UID: \"926e1eab-5155-41b4-8305-e95ce7e93417\") " Jul 14 18:06:24.087550 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.087510 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926e1eab-5155-41b4-8305-e95ce7e93417-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "926e1eab-5155-41b4-8305-e95ce7e93417" (UID: "926e1eab-5155-41b4-8305-e95ce7e93417"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:06:24.089291 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.089267 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926e1eab-5155-41b4-8305-e95ce7e93417-kube-api-access-hvzfm" (OuterVolumeSpecName: "kube-api-access-hvzfm") pod "926e1eab-5155-41b4-8305-e95ce7e93417" (UID: "926e1eab-5155-41b4-8305-e95ce7e93417"). InnerVolumeSpecName "kube-api-access-hvzfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:06:24.188015 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.187976 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/926e1eab-5155-41b4-8305-e95ce7e93417-extensions-socket-volume\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:06:24.188015 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.188006 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvzfm\" (UniqueName: \"kubernetes.io/projected/926e1eab-5155-41b4-8305-e95ce7e93417-kube-api-access-hvzfm\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:06:24.544645 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.544543 2572 generic.go:356] "Generic (PLEG): container finished" podID="926e1eab-5155-41b4-8305-e95ce7e93417" containerID="b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd" exitCode=0 Jul 14 18:06:24.544645 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.544598 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" Jul 14 18:06:24.544645 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.544628 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" event={"ID":"926e1eab-5155-41b4-8305-e95ce7e93417","Type":"ContainerDied","Data":"b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd"} Jul 14 18:06:24.544869 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.544715 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p" event={"ID":"926e1eab-5155-41b4-8305-e95ce7e93417","Type":"ContainerDied","Data":"cb32432bce23616a45f33a01947b76e25459dfe2915f6ac034d1a2b6dee8c633"} Jul 14 18:06:24.544869 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.544742 2572 scope.go:117] "RemoveContainer" containerID="b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd" Jul 14 18:06:24.554525 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.554501 2572 scope.go:117] "RemoveContainer" containerID="b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd" Jul 14 18:06:24.554823 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:24.554802 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd\": container with ID starting with b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd not found: ID does not exist" containerID="b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd" Jul 14 18:06:24.554898 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.554835 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd"} err="failed to get container status \"b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd\": rpc error: code = NotFound desc = could not find container \"b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd\": container with ID starting with b16d940f3d0032b45dc0ae3945dcad47b13245137df58050746cbad68c6facdd not found: ID does not exist" Jul 14 18:06:24.569655 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.569558 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p"] Jul 14 18:06:24.571374 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:24.571358 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xjs2p"] Jul 14 18:06:25.064103 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:25.064047 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:25.165887 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:25.165830 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:25.680724 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:25.680682 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:25.718077 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:25.718048 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="926e1eab-5155-41b4-8305-e95ce7e93417" path="/var/lib/kubelet/pods/926e1eab-5155-41b4-8305-e95ce7e93417/volumes" Jul 14 18:06:25.814199 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:25.814170 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:27.854125 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:27.854090 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:27.953953 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:27.953921 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:33.072810 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:33.072775 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:33.175526 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:33.175480 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:35.932948 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:35.932907 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:40.130035 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.129992 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:40.299784 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.299580 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:40.532222 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.532140 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6"] Jul 14 18:06:40.532522 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.532508 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926e1eab-5155-41b4-8305-e95ce7e93417" containerName="manager" Jul 14 18:06:40.532571 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.532524 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="926e1eab-5155-41b4-8305-e95ce7e93417" containerName="manager" Jul 14 18:06:40.532627 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.532594 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="926e1eab-5155-41b4-8305-e95ce7e93417" containerName="manager" Jul 14 18:06:40.588566 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.588537 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:40.730298 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.730264 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6"] Jul 14 18:06:40.730477 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.730373 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732406 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732538 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732418 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed28cebc-3944-46b3-92bf-946678bc3030-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732538 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732450 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed28cebc-3944-46b3-92bf-946678bc3030-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732538 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed28cebc-3944-46b3-92bf-946678bc3030-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732538 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xddx5\" (UniqueName: \"kubernetes.io/projected/ed28cebc-3944-46b3-92bf-946678bc3030-kube-api-access-xddx5\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732690 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732574 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732690 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732600 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732690 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.732690 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.732672 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.734546 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.734525 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 18:06:40.734683 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.734525 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ptvrr\"" Jul 14 18:06:40.734683 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.734595 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 18:06:40.734914 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.734898 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 18:06:40.833250 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833227 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed28cebc-3944-46b3-92bf-946678bc3030-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833257 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed28cebc-3944-46b3-92bf-946678bc3030-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xddx5\" (UniqueName: \"kubernetes.io/projected/ed28cebc-3944-46b3-92bf-946678bc3030-kube-api-access-xddx5\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833452 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833430 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833504 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833479 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833633 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833633 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833550 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833633 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833799 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833669 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed28cebc-3944-46b3-92bf-946678bc3030-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833799 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833782 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.833943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833925 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed28cebc-3944-46b3-92bf-946678bc3030-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.834010 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.833985 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.834067 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.834046 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.834124 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.834076 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.835106 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.835067 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:40.836227 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.836207 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed28cebc-3944-46b3-92bf-946678bc3030-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.836447 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.836430 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed28cebc-3944-46b3-92bf-946678bc3030-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.847822 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.847799 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed28cebc-3944-46b3-92bf-946678bc3030-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:40.847965 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:40.847950 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xddx5\" (UniqueName: \"kubernetes.io/projected/ed28cebc-3944-46b3-92bf-946678bc3030-kube-api-access-xddx5\") pod \"maas-default-gateway-openshift-default-56654dcf7-6t6m6\" (UID: \"ed28cebc-3944-46b3-92bf-946678bc3030\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:41.041357 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:41.041325 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:41.236667 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:41.236639 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6"] Jul 14 18:06:41.241782 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:06:41.239153 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded28cebc_3944_46b3_92bf_946678bc3030.slice/crio-d750fb6cb0adcb60f4658fde7ffffa81377e9d3d71feb39423004e62a0782a5c WatchSource:0}: Error finding container d750fb6cb0adcb60f4658fde7ffffa81377e9d3d71feb39423004e62a0782a5c: Status 404 returned error can't find the container with id d750fb6cb0adcb60f4658fde7ffffa81377e9d3d71feb39423004e62a0782a5c Jul 14 18:06:41.626714 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:41.626675 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" event={"ID":"ed28cebc-3944-46b3-92bf-946678bc3030","Type":"ContainerStarted","Data":"d750fb6cb0adcb60f4658fde7ffffa81377e9d3d71feb39423004e62a0782a5c"} Jul 14 18:06:44.383419 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:44.383377 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:06:44.383749 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:44.383450 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:06:44.383749 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:44.383477 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:06:44.644311 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:44.644219 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" event={"ID":"ed28cebc-3944-46b3-92bf-946678bc3030","Type":"ContainerStarted","Data":"56bdcfbd988cda0ec2ab405ec09ae4c62214573c43835cc20f75afd7eb7d813a"} Jul 14 18:06:44.673805 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:44.673754 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" podStartSLOduration=1.534454941 podStartE2EDuration="4.673737991s" totalImagesPullingTime="3.13928305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:44.67278619 +0000 UTC m=+747.535576030" watchObservedRunningTime="2026-07-14 18:06:44.673737991 +0000 UTC m=+747.536527795" Jul 14 18:06:45.041966 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.041888 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:45.046295 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.046271 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:45.649682 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.649649 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:45.650768 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.650746 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:06:45.890823 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.890792 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:06:45.900847 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.900777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:45.921395 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.921371 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:06:45.921557 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.921479 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:45.928686 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.928012 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 18:06:45.975871 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.975839 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgtk\" (UniqueName: \"kubernetes.io/projected/34b49289-d841-471c-bc14-a0a7d9d8c63e-kube-api-access-6qgtk\") pod \"limitador-limitador-64c94fd64f-cshdf\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:45.976039 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.975899 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b49289-d841-471c-bc14-a0a7d9d8c63e-config-file\") pod \"limitador-limitador-64c94fd64f-cshdf\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:45.981658 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:45.981633 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:06:46.021201 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.021174 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:46.076560 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.076525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b49289-d841-471c-bc14-a0a7d9d8c63e-config-file\") pod \"limitador-limitador-64c94fd64f-cshdf\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:46.076742 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.076654 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgtk\" (UniqueName: \"kubernetes.io/projected/34b49289-d841-471c-bc14-a0a7d9d8c63e-kube-api-access-6qgtk\") pod \"limitador-limitador-64c94fd64f-cshdf\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:46.077121 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.077100 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b49289-d841-471c-bc14-a0a7d9d8c63e-config-file\") pod \"limitador-limitador-64c94fd64f-cshdf\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:46.088755 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.088733 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgtk\" (UniqueName: \"kubernetes.io/projected/34b49289-d841-471c-bc14-a0a7d9d8c63e-kube-api-access-6qgtk\") pod \"limitador-limitador-64c94fd64f-cshdf\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:46.125672 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.125605 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:46.234023 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.233938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:46.377084 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.376984 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:06:46.379851 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:06:46.379819 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b49289_d841_471c_bc14_a0a7d9d8c63e.slice/crio-260daf1eac08750d77806c85c4ad254f5aea0382aa2bad04a589409e64e9983c WatchSource:0}: Error finding container 260daf1eac08750d77806c85c4ad254f5aea0382aa2bad04a589409e64e9983c: Status 404 returned error can't find the container with id 260daf1eac08750d77806c85c4ad254f5aea0382aa2bad04a589409e64e9983c Jul 14 18:06:46.656244 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.656204 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" event={"ID":"34b49289-d841-471c-bc14-a0a7d9d8c63e","Type":"ContainerStarted","Data":"260daf1eac08750d77806c85c4ad254f5aea0382aa2bad04a589409e64e9983c"} Jul 14 18:06:46.816330 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.816302 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-slhwm"] Jul 14 18:06:46.909876 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.909802 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-slhwm"] Jul 14 18:06:46.910012 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.909893 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:46.913767 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.913747 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l7nd9\"" Jul 14 18:06:46.984936 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:46.984910 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9k5v\" (UniqueName: \"kubernetes.io/projected/06bfd9e4-9fbe-4761-bad1-99eac2e365f9-kube-api-access-p9k5v\") pod \"authorino-58cd955675-slhwm\" (UID: \"06bfd9e4-9fbe-4761-bad1-99eac2e365f9\") " pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:47.086048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:47.086014 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9k5v\" (UniqueName: \"kubernetes.io/projected/06bfd9e4-9fbe-4761-bad1-99eac2e365f9-kube-api-access-p9k5v\") pod \"authorino-58cd955675-slhwm\" (UID: \"06bfd9e4-9fbe-4761-bad1-99eac2e365f9\") " pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:47.099662 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:47.099635 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9k5v\" (UniqueName: \"kubernetes.io/projected/06bfd9e4-9fbe-4761-bad1-99eac2e365f9-kube-api-access-p9k5v\") pod \"authorino-58cd955675-slhwm\" (UID: \"06bfd9e4-9fbe-4761-bad1-99eac2e365f9\") " pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:47.219685 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:47.219600 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:47.265641 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:47.265587 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:47.401996 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:47.401932 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-slhwm"] Jul 14 18:06:47.407091 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:06:47.406303 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06bfd9e4_9fbe_4761_bad1_99eac2e365f9.slice/crio-3e7fda4e91e52a146be3e36c78521702f4844703831f89e45d7474543ccdf7e7 WatchSource:0}: Error finding container 3e7fda4e91e52a146be3e36c78521702f4844703831f89e45d7474543ccdf7e7: Status 404 returned error can't find the container with id 3e7fda4e91e52a146be3e36c78521702f4844703831f89e45d7474543ccdf7e7 Jul 14 18:06:47.665220 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:47.665171 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-slhwm" event={"ID":"06bfd9e4-9fbe-4761-bad1-99eac2e365f9","Type":"ContainerStarted","Data":"3e7fda4e91e52a146be3e36c78521702f4844703831f89e45d7474543ccdf7e7"} Jul 14 18:06:48.444753 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:48.444714 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:48.553943 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:48.553903 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:06:50.893659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:50.893625 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-slhwm"] Jul 14 18:06:51.692774 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.692737 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" event={"ID":"34b49289-d841-471c-bc14-a0a7d9d8c63e","Type":"ContainerStarted","Data":"73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab"} Jul 14 18:06:51.692968 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.692893 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:06:51.694111 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.694090 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-slhwm" event={"ID":"06bfd9e4-9fbe-4761-bad1-99eac2e365f9","Type":"ContainerStarted","Data":"d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191"} Jul 14 18:06:51.694225 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.694153 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-slhwm" podUID="06bfd9e4-9fbe-4761-bad1-99eac2e365f9" containerName="authorino" containerID="cri-o://d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191" gracePeriod=30 Jul 14 18:06:51.716325 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.716280 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" podStartSLOduration=1.924651232 podStartE2EDuration="6.716263228s" totalImagesPullingTime="4.791611996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:51.713837516 +0000 UTC m=+754.576627323" watchObservedRunningTime="2026-07-14 18:06:51.716263228 +0000 UTC m=+754.579053032" Jul 14 18:06:51.733769 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.733727 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-slhwm" podStartSLOduration=1.913092193 podStartE2EDuration="5.733711741s" totalImagesPullingTime="3.820619548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:51.73239579 +0000 UTC m=+754.595185595" watchObservedRunningTime="2026-07-14 18:06:51.733711741 +0000 UTC m=+754.596501545" Jul 14 18:06:51.945102 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:51.945041 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:52.033663 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.033629 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9k5v\" (UniqueName: \"kubernetes.io/projected/06bfd9e4-9fbe-4761-bad1-99eac2e365f9-kube-api-access-p9k5v\") pod \"06bfd9e4-9fbe-4761-bad1-99eac2e365f9\" (UID: \"06bfd9e4-9fbe-4761-bad1-99eac2e365f9\") " Jul 14 18:06:52.038681 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.036627 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bfd9e4-9fbe-4761-bad1-99eac2e365f9-kube-api-access-p9k5v" (OuterVolumeSpecName: "kube-api-access-p9k5v") pod "06bfd9e4-9fbe-4761-bad1-99eac2e365f9" (UID: "06bfd9e4-9fbe-4761-bad1-99eac2e365f9"). InnerVolumeSpecName "kube-api-access-p9k5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:06:52.135669 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.135602 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9k5v\" (UniqueName: \"kubernetes.io/projected/06bfd9e4-9fbe-4761-bad1-99eac2e365f9-kube-api-access-p9k5v\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:06:52.700441 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.700403 2572 generic.go:356] "Generic (PLEG): container finished" podID="06bfd9e4-9fbe-4761-bad1-99eac2e365f9" containerID="d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191" exitCode=0 Jul 14 18:06:52.700626 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.700461 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-slhwm" Jul 14 18:06:52.700626 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.700499 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-slhwm" event={"ID":"06bfd9e4-9fbe-4761-bad1-99eac2e365f9","Type":"ContainerDied","Data":"d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191"} Jul 14 18:06:52.700626 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.700540 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-slhwm" event={"ID":"06bfd9e4-9fbe-4761-bad1-99eac2e365f9","Type":"ContainerDied","Data":"3e7fda4e91e52a146be3e36c78521702f4844703831f89e45d7474543ccdf7e7"} Jul 14 18:06:52.700626 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.700562 2572 scope.go:117] "RemoveContainer" containerID="d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191" Jul 14 18:06:52.709744 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.709726 2572 scope.go:117] "RemoveContainer" containerID="d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191" Jul 14 18:06:52.709999 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:06:52.709976 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191\": container with ID starting with d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191 not found: ID does not exist" containerID="d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191" Jul 14 18:06:52.710083 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.710003 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191"} err="failed to get container status \"d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191\": rpc error: code = NotFound desc = could not find container \"d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191\": container with ID starting with d0dbfaafa1b70f6b8c753d6c15f68eef4230ef9f4b34a1529e0e908915831191 not found: ID does not exist" Jul 14 18:06:52.723399 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.723371 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-slhwm"] Jul 14 18:06:52.725545 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:52.725525 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-slhwm"] Jul 14 18:06:53.720500 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:06:53.720460 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06bfd9e4-9fbe-4761-bad1-99eac2e365f9" path="/var/lib/kubelet/pods/06bfd9e4-9fbe-4761-bad1-99eac2e365f9/volumes" Jul 14 18:07:00.568633 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:00.568591 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:07:00.569077 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:00.568787 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" podUID="34b49289-d841-471c-bc14-a0a7d9d8c63e" containerName="limitador" containerID="cri-o://73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab" gracePeriod=30 Jul 14 18:07:00.569555 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:00.569532 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:07:01.129638 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.129550 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:07:01.215862 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.215831 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qgtk\" (UniqueName: \"kubernetes.io/projected/34b49289-d841-471c-bc14-a0a7d9d8c63e-kube-api-access-6qgtk\") pod \"34b49289-d841-471c-bc14-a0a7d9d8c63e\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " Jul 14 18:07:01.216035 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.215877 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b49289-d841-471c-bc14-a0a7d9d8c63e-config-file\") pod \"34b49289-d841-471c-bc14-a0a7d9d8c63e\" (UID: \"34b49289-d841-471c-bc14-a0a7d9d8c63e\") " Jul 14 18:07:01.216262 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.216234 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34b49289-d841-471c-bc14-a0a7d9d8c63e-config-file" (OuterVolumeSpecName: "config-file") pod "34b49289-d841-471c-bc14-a0a7d9d8c63e" (UID: "34b49289-d841-471c-bc14-a0a7d9d8c63e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:07:01.218094 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.218063 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34b49289-d841-471c-bc14-a0a7d9d8c63e-kube-api-access-6qgtk" (OuterVolumeSpecName: "kube-api-access-6qgtk") pod "34b49289-d841-471c-bc14-a0a7d9d8c63e" (UID: "34b49289-d841-471c-bc14-a0a7d9d8c63e"). InnerVolumeSpecName "kube-api-access-6qgtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:01.316844 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.316810 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qgtk\" (UniqueName: \"kubernetes.io/projected/34b49289-d841-471c-bc14-a0a7d9d8c63e-kube-api-access-6qgtk\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:07:01.316844 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.316837 2572 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34b49289-d841-471c-bc14-a0a7d9d8c63e-config-file\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:07:01.744050 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.744015 2572 generic.go:356] "Generic (PLEG): container finished" podID="34b49289-d841-471c-bc14-a0a7d9d8c63e" containerID="73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab" exitCode=0 Jul 14 18:07:01.744425 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.744064 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" Jul 14 18:07:01.744425 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.744090 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" event={"ID":"34b49289-d841-471c-bc14-a0a7d9d8c63e","Type":"ContainerDied","Data":"73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab"} Jul 14 18:07:01.744425 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.744141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cshdf" event={"ID":"34b49289-d841-471c-bc14-a0a7d9d8c63e","Type":"ContainerDied","Data":"260daf1eac08750d77806c85c4ad254f5aea0382aa2bad04a589409e64e9983c"} Jul 14 18:07:01.744425 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.744171 2572 scope.go:117] "RemoveContainer" containerID="73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab" Jul 14 18:07:01.752561 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.752542 2572 scope.go:117] "RemoveContainer" containerID="73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab" Jul 14 18:07:01.752835 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:07:01.752815 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab\": container with ID starting with 73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab not found: ID does not exist" containerID="73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab" Jul 14 18:07:01.752900 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.752848 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab"} err="failed to get container status \"73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab\": rpc error: code = NotFound desc = could not find container \"73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab\": container with ID starting with 73bb413ea62c0e5c36578606cb536c8a23500ee2437e948c63557e35376053ab not found: ID does not exist" Jul 14 18:07:01.771357 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.771333 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:07:01.779174 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:01.779154 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cshdf"] Jul 14 18:07:02.101418 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.101380 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:02.300820 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.300784 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:02.607547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607520 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq"] Jul 14 18:07:02.607911 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607898 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34b49289-d841-471c-bc14-a0a7d9d8c63e" containerName="limitador" Jul 14 18:07:02.607955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607913 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="34b49289-d841-471c-bc14-a0a7d9d8c63e" containerName="limitador" Jul 14 18:07:02.607955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607932 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bfd9e4-9fbe-4761-bad1-99eac2e365f9" containerName="authorino" Jul 14 18:07:02.607955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607937 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd9e4-9fbe-4761-bad1-99eac2e365f9" containerName="authorino" Jul 14 18:07:02.608045 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607984 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="06bfd9e4-9fbe-4761-bad1-99eac2e365f9" containerName="authorino" Jul 14 18:07:02.608045 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.607996 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="34b49289-d841-471c-bc14-a0a7d9d8c63e" containerName="limitador" Jul 14 18:07:02.612141 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.612126 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.615312 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.615281 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 18:07:02.615444 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.615393 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 14 18:07:02.615935 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.615917 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-sb6t5\"" Jul 14 18:07:02.616014 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.615997 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 18:07:02.632461 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.632438 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq"] Jul 14 18:07:02.663693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.663658 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:02.730084 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.730046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f92f4bee-41a4-4add-8257-baa8ea9632ac-data\") pod \"postgres-69fb4594cf-dnrnq\" (UID: \"f92f4bee-41a4-4add-8257-baa8ea9632ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.730252 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.730110 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctkq8\" (UniqueName: \"kubernetes.io/projected/f92f4bee-41a4-4add-8257-baa8ea9632ac-kube-api-access-ctkq8\") pod \"postgres-69fb4594cf-dnrnq\" (UID: \"f92f4bee-41a4-4add-8257-baa8ea9632ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.830749 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.830709 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f92f4bee-41a4-4add-8257-baa8ea9632ac-data\") pod \"postgres-69fb4594cf-dnrnq\" (UID: \"f92f4bee-41a4-4add-8257-baa8ea9632ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.831125 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.830792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctkq8\" (UniqueName: \"kubernetes.io/projected/f92f4bee-41a4-4add-8257-baa8ea9632ac-kube-api-access-ctkq8\") pod \"postgres-69fb4594cf-dnrnq\" (UID: \"f92f4bee-41a4-4add-8257-baa8ea9632ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.831171 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.831149 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f92f4bee-41a4-4add-8257-baa8ea9632ac-data\") pod \"postgres-69fb4594cf-dnrnq\" (UID: \"f92f4bee-41a4-4add-8257-baa8ea9632ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.857013 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.856989 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctkq8\" (UniqueName: \"kubernetes.io/projected/f92f4bee-41a4-4add-8257-baa8ea9632ac-kube-api-access-ctkq8\") pod \"postgres-69fb4594cf-dnrnq\" (UID: \"f92f4bee-41a4-4add-8257-baa8ea9632ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:02.922787 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:02.922707 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:03.068336 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:03.068297 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq"] Jul 14 18:07:03.072724 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:03.072699 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf92f4bee_41a4_4add_8257_baa8ea9632ac.slice/crio-3944a3463799db10f246c0188850a9bf082d862f006e7b3047aa19b58554b69f WatchSource:0}: Error finding container 3944a3463799db10f246c0188850a9bf082d862f006e7b3047aa19b58554b69f: Status 404 returned error can't find the container with id 3944a3463799db10f246c0188850a9bf082d862f006e7b3047aa19b58554b69f Jul 14 18:07:03.721594 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:03.721558 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34b49289-d841-471c-bc14-a0a7d9d8c63e" path="/var/lib/kubelet/pods/34b49289-d841-471c-bc14-a0a7d9d8c63e/volumes" Jul 14 18:07:03.756433 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:03.756381 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" event={"ID":"f92f4bee-41a4-4add-8257-baa8ea9632ac","Type":"ContainerStarted","Data":"3944a3463799db10f246c0188850a9bf082d862f006e7b3047aa19b58554b69f"} Jul 14 18:07:07.780964 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:07.780926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" event={"ID":"f92f4bee-41a4-4add-8257-baa8ea9632ac","Type":"ContainerStarted","Data":"54dac4dd6e383e2e4f4e305840245cc7e198eda3912eb6073dd28b3709f97c86"} Jul 14 18:07:07.781360 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:07.781027 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:07.807076 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:07.807024 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" podStartSLOduration=1.250365027 podStartE2EDuration="5.807007698s" totalImagesPullingTime="4.556642671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:07.80454138 +0000 UTC m=+770.667331183" watchObservedRunningTime="2026-07-14 18:07:07.807007698 +0000 UTC m=+770.669797501" Jul 14 18:07:13.817000 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:13.816970 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dnrnq" Jul 14 18:07:14.153595 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.153512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:14.390827 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.390793 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4sx"] Jul 14 18:07:14.399423 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.399395 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:14.402534 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.402515 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l7nd9\"" Jul 14 18:07:14.409659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.409639 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4sx"] Jul 14 18:07:14.543885 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.543833 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:14.544057 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.544041 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85r7m\" (UniqueName: \"kubernetes.io/projected/d5168974-a515-4cde-b082-c161f7dd3fd5-kube-api-access-85r7m\") pod \"authorino-5d4b67c446-lk4sx\" (UID: \"d5168974-a515-4cde-b082-c161f7dd3fd5\") " pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:14.645083 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.645050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85r7m\" (UniqueName: \"kubernetes.io/projected/d5168974-a515-4cde-b082-c161f7dd3fd5-kube-api-access-85r7m\") pod \"authorino-5d4b67c446-lk4sx\" (UID: \"d5168974-a515-4cde-b082-c161f7dd3fd5\") " pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:14.658187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.658118 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85r7m\" (UniqueName: \"kubernetes.io/projected/d5168974-a515-4cde-b082-c161f7dd3fd5-kube-api-access-85r7m\") pod \"authorino-5d4b67c446-lk4sx\" (UID: \"d5168974-a515-4cde-b082-c161f7dd3fd5\") " pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:14.698740 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.698705 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4sx"] Jul 14 18:07:14.698992 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.698980 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:14.730344 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.730315 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-677cd69684-z8tdp"] Jul 14 18:07:14.734269 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.734245 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:14.749390 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.749321 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-677cd69684-z8tdp"] Jul 14 18:07:14.844733 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.844707 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4sx"] Jul 14 18:07:14.846634 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.846582 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-677cd69684-z8tdp"] Jul 14 18:07:14.846844 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.846822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f54rc\" (UniqueName: \"kubernetes.io/projected/7d3ff145-ad01-4656-928a-fed64cd014b3-kube-api-access-f54rc\") pod \"authorino-677cd69684-z8tdp\" (UID: \"7d3ff145-ad01-4656-928a-fed64cd014b3\") " pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:14.846910 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:07:14.846875 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-f54rc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-677cd69684-z8tdp" podUID="7d3ff145-ad01-4656-928a-fed64cd014b3" Jul 14 18:07:14.847137 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:14.847116 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5168974_a515_4cde_b082_c161f7dd3fd5.slice/crio-8baaa94c6b322054b32f5a51a06fe6b696accafe3be6b7428f7a2511fade4e6a WatchSource:0}: Error finding container 8baaa94c6b322054b32f5a51a06fe6b696accafe3be6b7428f7a2511fade4e6a: Status 404 returned error can't find the container with id 8baaa94c6b322054b32f5a51a06fe6b696accafe3be6b7428f7a2511fade4e6a Jul 14 18:07:14.948219 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.948145 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f54rc\" (UniqueName: \"kubernetes.io/projected/7d3ff145-ad01-4656-928a-fed64cd014b3-kube-api-access-f54rc\") pod \"authorino-677cd69684-z8tdp\" (UID: \"7d3ff145-ad01-4656-928a-fed64cd014b3\") " pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:14.961182 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:14.961157 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f54rc\" (UniqueName: \"kubernetes.io/projected/7d3ff145-ad01-4656-928a-fed64cd014b3-kube-api-access-f54rc\") pod \"authorino-677cd69684-z8tdp\" (UID: \"7d3ff145-ad01-4656-928a-fed64cd014b3\") " pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:15.820482 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.820448 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:15.820707 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.820489 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" event={"ID":"d5168974-a515-4cde-b082-c161f7dd3fd5","Type":"ContainerStarted","Data":"dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e"} Jul 14 18:07:15.820707 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.820509 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" podUID="d5168974-a515-4cde-b082-c161f7dd3fd5" containerName="authorino" containerID="cri-o://dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e" gracePeriod=30 Jul 14 18:07:15.820707 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.820535 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" event={"ID":"d5168974-a515-4cde-b082-c161f7dd3fd5","Type":"ContainerStarted","Data":"8baaa94c6b322054b32f5a51a06fe6b696accafe3be6b7428f7a2511fade4e6a"} Jul 14 18:07:15.827486 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.827469 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:15.866247 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.866198 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" podStartSLOduration=1.053696807 podStartE2EDuration="1.866180885s" totalImagesPullingTime="812.484078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:15.863802632 +0000 UTC m=+778.726592429" watchObservedRunningTime="2026-07-14 18:07:15.866180885 +0000 UTC m=+778.728970689" Jul 14 18:07:15.956188 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.956159 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f54rc\" (UniqueName: \"kubernetes.io/projected/7d3ff145-ad01-4656-928a-fed64cd014b3-kube-api-access-f54rc\") pod \"7d3ff145-ad01-4656-928a-fed64cd014b3\" (UID: \"7d3ff145-ad01-4656-928a-fed64cd014b3\") " Jul 14 18:07:15.958272 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:15.958237 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3ff145-ad01-4656-928a-fed64cd014b3-kube-api-access-f54rc" (OuterVolumeSpecName: "kube-api-access-f54rc") pod "7d3ff145-ad01-4656-928a-fed64cd014b3" (UID: "7d3ff145-ad01-4656-928a-fed64cd014b3"). InnerVolumeSpecName "kube-api-access-f54rc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:16.056859 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.056828 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f54rc\" (UniqueName: \"kubernetes.io/projected/7d3ff145-ad01-4656-928a-fed64cd014b3-kube-api-access-f54rc\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:07:16.084318 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.084290 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:16.157547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.157510 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85r7m\" (UniqueName: \"kubernetes.io/projected/d5168974-a515-4cde-b082-c161f7dd3fd5-kube-api-access-85r7m\") pod \"d5168974-a515-4cde-b082-c161f7dd3fd5\" (UID: \"d5168974-a515-4cde-b082-c161f7dd3fd5\") " Jul 14 18:07:16.159802 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.159777 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5168974-a515-4cde-b082-c161f7dd3fd5-kube-api-access-85r7m" (OuterVolumeSpecName: "kube-api-access-85r7m") pod "d5168974-a515-4cde-b082-c161f7dd3fd5" (UID: "d5168974-a515-4cde-b082-c161f7dd3fd5"). InnerVolumeSpecName "kube-api-access-85r7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:16.259070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.258985 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85r7m\" (UniqueName: \"kubernetes.io/projected/d5168974-a515-4cde-b082-c161f7dd3fd5-kube-api-access-85r7m\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:07:16.826491 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.826461 2572 generic.go:356] "Generic (PLEG): container finished" podID="d5168974-a515-4cde-b082-c161f7dd3fd5" containerID="dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e" exitCode=0 Jul 14 18:07:16.826727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.826507 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" Jul 14 18:07:16.826727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.826550 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" event={"ID":"d5168974-a515-4cde-b082-c161f7dd3fd5","Type":"ContainerDied","Data":"dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e"} Jul 14 18:07:16.826727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.826600 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lk4sx" event={"ID":"d5168974-a515-4cde-b082-c161f7dd3fd5","Type":"ContainerDied","Data":"8baaa94c6b322054b32f5a51a06fe6b696accafe3be6b7428f7a2511fade4e6a"} Jul 14 18:07:16.826727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.826641 2572 scope.go:117] "RemoveContainer" containerID="dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e" Jul 14 18:07:16.826907 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.826620 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-677cd69684-z8tdp" Jul 14 18:07:16.835681 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.835669 2572 scope.go:117] "RemoveContainer" containerID="dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e" Jul 14 18:07:16.835922 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:07:16.835907 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e\": container with ID starting with dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e not found: ID does not exist" containerID="dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e" Jul 14 18:07:16.835984 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.835929 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e"} err="failed to get container status \"dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e\": rpc error: code = NotFound desc = could not find container \"dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e\": container with ID starting with dcd4f16616852bd594ec10c97c9bfcfc48eebec4d8be4918eecdfb24dad1322e not found: ID does not exist" Jul 14 18:07:16.904713 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.904668 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-677cd69684-z8tdp"] Jul 14 18:07:16.916711 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.916682 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-677cd69684-z8tdp"] Jul 14 18:07:16.947206 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.947137 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4sx"] Jul 14 18:07:16.963736 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:16.963704 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lk4sx"] Jul 14 18:07:17.718981 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:17.718952 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d3ff145-ad01-4656-928a-fed64cd014b3" path="/var/lib/kubelet/pods/7d3ff145-ad01-4656-928a-fed64cd014b3/volumes" Jul 14 18:07:17.719178 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:17.719165 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5168974-a515-4cde-b082-c161f7dd3fd5" path="/var/lib/kubelet/pods/d5168974-a515-4cde-b082-c161f7dd3fd5/volumes" Jul 14 18:07:21.231562 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.231522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:21.263010 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.262979 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-88d46f585-24njc"] Jul 14 18:07:21.263349 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.263337 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5168974-a515-4cde-b082-c161f7dd3fd5" containerName="authorino" Jul 14 18:07:21.263394 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.263350 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5168974-a515-4cde-b082-c161f7dd3fd5" containerName="authorino" Jul 14 18:07:21.263427 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.263410 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5168974-a515-4cde-b082-c161f7dd3fd5" containerName="authorino" Jul 14 18:07:21.338087 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.338055 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88d46f585-24njc"] Jul 14 18:07:21.338246 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.338166 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:21.342666 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.342638 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 18:07:21.342796 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.342756 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bj86r\"" Jul 14 18:07:21.343552 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.343533 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 18:07:21.380026 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.379996 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:21.479367 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.479334 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-4tv4t"] Jul 14 18:07:21.499400 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.499307 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:21.502771 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.502749 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-4tv4t"] Jul 14 18:07:21.502881 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.502853 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.505990 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.505967 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:21.506175 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.505998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w98sj\" (UniqueName: \"kubernetes.io/projected/3a08f395-3797-400d-aaaf-eedb859acacc-kube-api-access-w98sj\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:21.606574 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.606542 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67404d70-da1c-4223-8636-bfa02ecc656b-webhook-cert\") pod \"maas-controller-5f4bb644f5-4tv4t\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.606753 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.606603 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:21.606753 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.606688 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w98sj\" (UniqueName: \"kubernetes.io/projected/3a08f395-3797-400d-aaaf-eedb859acacc-kube-api-access-w98sj\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:21.606753 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:07:21.606738 2572 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 14 18:07:21.606888 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.606803 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6s8x\" (UniqueName: \"kubernetes.io/projected/67404d70-da1c-4223-8636-bfa02ecc656b-kube-api-access-k6s8x\") pod \"maas-controller-5f4bb644f5-4tv4t\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.606888 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:07:21.606814 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert podName:3a08f395-3797-400d-aaaf-eedb859acacc nodeName:}" failed. No retries permitted until 2026-07-14 18:07:22.106796901 +0000 UTC m=+784.969586682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert") pod "maas-controller-88d46f585-24njc" (UID: "3a08f395-3797-400d-aaaf-eedb859acacc") : secret "maas-controller-webhook-cert" not found Jul 14 18:07:21.616595 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.616572 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w98sj\" (UniqueName: \"kubernetes.io/projected/3a08f395-3797-400d-aaaf-eedb859acacc-kube-api-access-w98sj\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:21.707633 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.707580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67404d70-da1c-4223-8636-bfa02ecc656b-webhook-cert\") pod \"maas-controller-5f4bb644f5-4tv4t\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.707824 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.707708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6s8x\" (UniqueName: \"kubernetes.io/projected/67404d70-da1c-4223-8636-bfa02ecc656b-kube-api-access-k6s8x\") pod \"maas-controller-5f4bb644f5-4tv4t\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.710222 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.710194 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67404d70-da1c-4223-8636-bfa02ecc656b-webhook-cert\") pod \"maas-controller-5f4bb644f5-4tv4t\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.721216 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.721188 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6s8x\" (UniqueName: \"kubernetes.io/projected/67404d70-da1c-4223-8636-bfa02ecc656b-kube-api-access-k6s8x\") pod \"maas-controller-5f4bb644f5-4tv4t\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.760307 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.760220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:21.814013 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.813980 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:21.975980 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:21.975944 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-4tv4t"] Jul 14 18:07:21.980024 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:21.979996 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67404d70_da1c_4223_8636_bfa02ecc656b.slice/crio-abbc691936de6141a10ac36ba54f04f74a83145ee71700b6af00aa38bf179ce1 WatchSource:0}: Error finding container abbc691936de6141a10ac36ba54f04f74a83145ee71700b6af00aa38bf179ce1: Status 404 returned error can't find the container with id abbc691936de6141a10ac36ba54f04f74a83145ee71700b6af00aa38bf179ce1 Jul 14 18:07:22.112190 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:22.112151 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:22.114674 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:22.114655 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert\") pod \"maas-controller-88d46f585-24njc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:22.249032 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:22.249000 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:22.411104 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:22.411080 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88d46f585-24njc"] Jul 14 18:07:22.412648 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:22.412625 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a08f395_3797_400d_aaaf_eedb859acacc.slice/crio-adc2b96d39d493da8d1bffeacfcb155d32fdf46b7a7460a4e622dc1ee9dd65c8 WatchSource:0}: Error finding container adc2b96d39d493da8d1bffeacfcb155d32fdf46b7a7460a4e622dc1ee9dd65c8: Status 404 returned error can't find the container with id adc2b96d39d493da8d1bffeacfcb155d32fdf46b7a7460a4e622dc1ee9dd65c8 Jul 14 18:07:22.863055 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:22.863013 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" event={"ID":"67404d70-da1c-4223-8636-bfa02ecc656b","Type":"ContainerStarted","Data":"abbc691936de6141a10ac36ba54f04f74a83145ee71700b6af00aa38bf179ce1"} Jul 14 18:07:22.864759 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:22.864729 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88d46f585-24njc" event={"ID":"3a08f395-3797-400d-aaaf-eedb859acacc","Type":"ContainerStarted","Data":"adc2b96d39d493da8d1bffeacfcb155d32fdf46b7a7460a4e622dc1ee9dd65c8"} Jul 14 18:07:24.677472 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.677380 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:24.876848 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.876809 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" event={"ID":"67404d70-da1c-4223-8636-bfa02ecc656b","Type":"ContainerStarted","Data":"654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020"} Jul 14 18:07:24.877010 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.876940 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:24.878309 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.878283 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88d46f585-24njc" event={"ID":"3a08f395-3797-400d-aaaf-eedb859acacc","Type":"ContainerStarted","Data":"e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb"} Jul 14 18:07:24.878440 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.878409 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:24.941255 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.941159 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" podStartSLOduration=1.52711714 podStartE2EDuration="3.941143751s" totalImagesPullingTime="2.414026611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:24.940766584 +0000 UTC m=+787.803556387" watchObservedRunningTime="2026-07-14 18:07:24.941143751 +0000 UTC m=+787.803933555" Jul 14 18:07:24.997441 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:24.997391 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-88d46f585-24njc" podStartSLOduration=2.01736641 podStartE2EDuration="3.997375219s" totalImagesPullingTime="1.980008809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:24.996836664 +0000 UTC m=+787.859626469" watchObservedRunningTime="2026-07-14 18:07:24.997375219 +0000 UTC m=+787.860165022" Jul 14 18:07:25.015639 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:25.015584 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:35.884384 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:35.884354 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:35.884807 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:35.884462 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:07:35.961168 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:35.961132 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-88d46f585-24njc"] Jul 14 18:07:35.961362 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:35.961336 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-88d46f585-24njc" podUID="3a08f395-3797-400d-aaaf-eedb859acacc" containerName="manager" containerID="cri-o://e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb" gracePeriod=30 Jul 14 18:07:36.204849 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.204824 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:36.241579 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.241552 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w98sj\" (UniqueName: \"kubernetes.io/projected/3a08f395-3797-400d-aaaf-eedb859acacc-kube-api-access-w98sj\") pod \"3a08f395-3797-400d-aaaf-eedb859acacc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " Jul 14 18:07:36.241756 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.241627 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert\") pod \"3a08f395-3797-400d-aaaf-eedb859acacc\" (UID: \"3a08f395-3797-400d-aaaf-eedb859acacc\") " Jul 14 18:07:36.243959 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.243925 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a08f395-3797-400d-aaaf-eedb859acacc-kube-api-access-w98sj" (OuterVolumeSpecName: "kube-api-access-w98sj") pod "3a08f395-3797-400d-aaaf-eedb859acacc" (UID: "3a08f395-3797-400d-aaaf-eedb859acacc"). InnerVolumeSpecName "kube-api-access-w98sj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:36.243959 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.243927 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3a08f395-3797-400d-aaaf-eedb859acacc" (UID: "3a08f395-3797-400d-aaaf-eedb859acacc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:07:36.342550 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.342519 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w98sj\" (UniqueName: \"kubernetes.io/projected/3a08f395-3797-400d-aaaf-eedb859acacc-kube-api-access-w98sj\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:07:36.342550 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.342547 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a08f395-3797-400d-aaaf-eedb859acacc-webhook-cert\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:07:36.941823 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.941790 2572 generic.go:356] "Generic (PLEG): container finished" podID="3a08f395-3797-400d-aaaf-eedb859acacc" containerID="e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb" exitCode=0 Jul 14 18:07:36.942221 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.941846 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88d46f585-24njc" Jul 14 18:07:36.942221 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.941847 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88d46f585-24njc" event={"ID":"3a08f395-3797-400d-aaaf-eedb859acacc","Type":"ContainerDied","Data":"e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb"} Jul 14 18:07:36.942221 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.941910 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88d46f585-24njc" event={"ID":"3a08f395-3797-400d-aaaf-eedb859acacc","Type":"ContainerDied","Data":"adc2b96d39d493da8d1bffeacfcb155d32fdf46b7a7460a4e622dc1ee9dd65c8"} Jul 14 18:07:36.942221 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.941938 2572 scope.go:117] "RemoveContainer" containerID="e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb" Jul 14 18:07:36.952907 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.952885 2572 scope.go:117] "RemoveContainer" containerID="e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb" Jul 14 18:07:36.953432 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:07:36.953410 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb\": container with ID starting with e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb not found: ID does not exist" containerID="e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb" Jul 14 18:07:36.953540 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.953439 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb"} err="failed to get container status \"e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb\": rpc error: code = NotFound desc = could not find container \"e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb\": container with ID starting with e6882d101b053d01ca430073b82cd782825b6a09794188bcfe0a12f9b6e4b0fb not found: ID does not exist" Jul 14 18:07:36.966190 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.966166 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-88d46f585-24njc"] Jul 14 18:07:36.971210 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:36.971190 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-88d46f585-24njc"] Jul 14 18:07:37.718533 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:37.718505 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a08f395-3797-400d-aaaf-eedb859acacc" path="/var/lib/kubelet/pods/3a08f395-3797-400d-aaaf-eedb859acacc/volumes" Jul 14 18:07:37.992640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:37.992542 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:56.674963 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:56.674863 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:56.835166 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:56.835128 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:57.251189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.251153 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:57.359334 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.359298 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:57.370579 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.370551 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp"] Jul 14 18:07:57.370973 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.370960 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a08f395-3797-400d-aaaf-eedb859acacc" containerName="manager" Jul 14 18:07:57.371017 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.370976 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a08f395-3797-400d-aaaf-eedb859acacc" containerName="manager" Jul 14 18:07:57.371063 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.371053 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a08f395-3797-400d-aaaf-eedb859acacc" containerName="manager" Jul 14 18:07:57.375890 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.375869 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.380155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.380129 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 14 18:07:57.380155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.380127 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\"" Jul 14 18:07:57.395423 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.395379 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp"] Jul 14 18:07:57.528557 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.525189 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/631cbb70-13c3-4850-83a9-c056bf670e27-maas-api-tls\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.528557 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.525270 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9rl4\" (UniqueName: \"kubernetes.io/projected/631cbb70-13c3-4850-83a9-c056bf670e27-kube-api-access-s9rl4\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.528557 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.525316 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/631cbb70-13c3-4850-83a9-c056bf670e27-openshift-service-ca\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.626739 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.626704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/631cbb70-13c3-4850-83a9-c056bf670e27-openshift-service-ca\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.626898 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.626808 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/631cbb70-13c3-4850-83a9-c056bf670e27-maas-api-tls\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.626898 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.626867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9rl4\" (UniqueName: \"kubernetes.io/projected/631cbb70-13c3-4850-83a9-c056bf670e27-kube-api-access-s9rl4\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.627482 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.627454 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/631cbb70-13c3-4850-83a9-c056bf670e27-openshift-service-ca\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.629512 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.629486 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/631cbb70-13c3-4850-83a9-c056bf670e27-maas-api-tls\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.636176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.636151 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9rl4\" (UniqueName: \"kubernetes.io/projected/631cbb70-13c3-4850-83a9-c056bf670e27-kube-api-access-s9rl4\") pod \"maas-api-67bf4f8764-8nhhp\" (UID: \"631cbb70-13c3-4850-83a9-c056bf670e27\") " pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.685823 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.685801 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:07:57.835279 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:57.835250 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp"] Jul 14 18:07:57.837155 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:57.837127 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631cbb70_13c3_4850_83a9_c056bf670e27.slice/crio-9410b2d050c1025553e2fe45ad7e8e6ce1d836592bba9357efcfc86abc701956 WatchSource:0}: Error finding container 9410b2d050c1025553e2fe45ad7e8e6ce1d836592bba9357efcfc86abc701956: Status 404 returned error can't find the container with id 9410b2d050c1025553e2fe45ad7e8e6ce1d836592bba9357efcfc86abc701956 Jul 14 18:07:58.026433 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.026390 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:58.042523 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.042494 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" event={"ID":"631cbb70-13c3-4850-83a9-c056bf670e27","Type":"ContainerStarted","Data":"9410b2d050c1025553e2fe45ad7e8e6ce1d836592bba9357efcfc86abc701956"} Jul 14 18:07:58.052158 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.052128 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-httf6"] Jul 14 18:07:58.055547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.055532 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.058720 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.058697 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 18:07:58.072349 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.072329 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-httf6"] Jul 14 18:07:58.130921 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.130847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8chw\" (UniqueName: \"kubernetes.io/projected/6108e278-8882-49da-9e5a-8e016c95b5a0-kube-api-access-q8chw\") pod \"payload-pre-processing-74584bb9f8-httf6\" (UID: \"6108e278-8882-49da-9e5a-8e016c95b5a0\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.130921 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.130904 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108e278-8882-49da-9e5a-8e016c95b5a0-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-httf6\" (UID: \"6108e278-8882-49da-9e5a-8e016c95b5a0\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.148176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.148140 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:07:58.189245 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.189219 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-kvwjb"] Jul 14 18:07:58.191685 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.191669 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.195433 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.195414 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zs54s\"" Jul 14 18:07:58.205766 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.205743 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-kvwjb"] Jul 14 18:07:58.232144 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.232118 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8chw\" (UniqueName: \"kubernetes.io/projected/6108e278-8882-49da-9e5a-8e016c95b5a0-kube-api-access-q8chw\") pod \"payload-pre-processing-74584bb9f8-httf6\" (UID: \"6108e278-8882-49da-9e5a-8e016c95b5a0\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.232292 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.232195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108e278-8882-49da-9e5a-8e016c95b5a0-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-httf6\" (UID: \"6108e278-8882-49da-9e5a-8e016c95b5a0\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.232992 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.232970 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108e278-8882-49da-9e5a-8e016c95b5a0-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-httf6\" (UID: \"6108e278-8882-49da-9e5a-8e016c95b5a0\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.241435 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.241413 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8chw\" (UniqueName: \"kubernetes.io/projected/6108e278-8882-49da-9e5a-8e016c95b5a0-kube-api-access-q8chw\") pod \"payload-pre-processing-74584bb9f8-httf6\" (UID: \"6108e278-8882-49da-9e5a-8e016c95b5a0\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.332724 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.332686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c1ba809-dcf1-451c-b202-a01858d6d106-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-kvwjb\" (UID: \"2c1ba809-dcf1-451c-b202-a01858d6d106\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.332724 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.332728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpsxz\" (UniqueName: \"kubernetes.io/projected/2c1ba809-dcf1-451c-b202-a01858d6d106-kube-api-access-jpsxz\") pod \"payload-processing-f9b8d9b9b-kvwjb\" (UID: \"2c1ba809-dcf1-451c-b202-a01858d6d106\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.366043 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.366015 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:07:58.433723 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.433693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c1ba809-dcf1-451c-b202-a01858d6d106-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-kvwjb\" (UID: \"2c1ba809-dcf1-451c-b202-a01858d6d106\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.433867 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.433750 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpsxz\" (UniqueName: \"kubernetes.io/projected/2c1ba809-dcf1-451c-b202-a01858d6d106-kube-api-access-jpsxz\") pod \"payload-processing-f9b8d9b9b-kvwjb\" (UID: \"2c1ba809-dcf1-451c-b202-a01858d6d106\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.434601 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.434575 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c1ba809-dcf1-451c-b202-a01858d6d106-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-kvwjb\" (UID: \"2c1ba809-dcf1-451c-b202-a01858d6d106\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.447132 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.447111 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpsxz\" (UniqueName: \"kubernetes.io/projected/2c1ba809-dcf1-451c-b202-a01858d6d106-kube-api-access-jpsxz\") pod \"payload-processing-f9b8d9b9b-kvwjb\" (UID: \"2c1ba809-dcf1-451c-b202-a01858d6d106\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.505345 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.505310 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:07:58.516939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.516912 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-httf6"] Jul 14 18:07:58.519625 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:58.519585 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6108e278_8882_49da_9e5a_8e016c95b5a0.slice/crio-12fdd1fed4427a1a6259ce96d63c45f2454ccd36a22a72b2e83a8f73b7f8f11f WatchSource:0}: Error finding container 12fdd1fed4427a1a6259ce96d63c45f2454ccd36a22a72b2e83a8f73b7f8f11f: Status 404 returned error can't find the container with id 12fdd1fed4427a1a6259ce96d63c45f2454ccd36a22a72b2e83a8f73b7f8f11f Jul 14 18:07:58.698299 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:58.698278 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-kvwjb"] Jul 14 18:07:58.700418 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:07:58.700387 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1ba809_dcf1_451c_b202_a01858d6d106.slice/crio-c4ee9d3a8df434650046fe09026211a755dc41f01bfb348197501be0f59e8cba WatchSource:0}: Error finding container c4ee9d3a8df434650046fe09026211a755dc41f01bfb348197501be0f59e8cba: Status 404 returned error can't find the container with id c4ee9d3a8df434650046fe09026211a755dc41f01bfb348197501be0f59e8cba Jul 14 18:07:59.052365 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:59.052324 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" event={"ID":"2c1ba809-dcf1-451c-b202-a01858d6d106","Type":"ContainerStarted","Data":"c4ee9d3a8df434650046fe09026211a755dc41f01bfb348197501be0f59e8cba"} Jul 14 18:07:59.053830 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:07:59.053795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" event={"ID":"6108e278-8882-49da-9e5a-8e016c95b5a0","Type":"ContainerStarted","Data":"12fdd1fed4427a1a6259ce96d63c45f2454ccd36a22a72b2e83a8f73b7f8f11f"} Jul 14 18:08:01.076603 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:01.076511 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" event={"ID":"2c1ba809-dcf1-451c-b202-a01858d6d106","Type":"ContainerStarted","Data":"5a18ec0359fccddf24cb53ba029bba50f89a13aa8b9ab583bb4aa71824f6b941"} Jul 14 18:08:01.077081 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:01.076673 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:08:01.078358 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:01.078337 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" event={"ID":"6108e278-8882-49da-9e5a-8e016c95b5a0","Type":"ContainerStarted","Data":"08104888b056103678c5e1b087f2ee00e19a1f4bd4fd9833d5706a56d712e655"} Jul 14 18:08:01.078463 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:01.078430 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:08:01.101125 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:01.101061 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" podStartSLOduration=1.469322475 podStartE2EDuration="3.101039045s" totalImagesPullingTime="1.63171657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:01.098069349 +0000 UTC m=+823.960859152" watchObservedRunningTime="2026-07-14 18:08:01.101039045 +0000 UTC m=+823.963828851" Jul 14 18:08:01.125671 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:01.125595 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" podStartSLOduration=1.315512583 podStartE2EDuration="3.125575733s" totalImagesPullingTime="1.81006315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:01.123246743 +0000 UTC m=+823.986036547" watchObservedRunningTime="2026-07-14 18:08:01.125575733 +0000 UTC m=+823.988365536" Jul 14 18:08:02.085141 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:02.085100 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" event={"ID":"631cbb70-13c3-4850-83a9-c056bf670e27","Type":"ContainerStarted","Data":"fabdd260dfb8c7c636b34b06abadf6d9f27135b7d3d3e0d524c4f851d8c96914"} Jul 14 18:08:02.085640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:02.085259 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:08:02.117031 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:02.116989 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" podStartSLOduration=1.733097406 podStartE2EDuration="5.116976064s" totalImagesPullingTime="3.383878658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:02.115233855 +0000 UTC m=+824.978023659" watchObservedRunningTime="2026-07-14 18:08:02.116976064 +0000 UTC m=+824.979765868" Jul 14 18:08:06.109152 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:06.109123 2572 generic.go:356] "Generic (PLEG): container finished" podID="ed28cebc-3944-46b3-92bf-946678bc3030" containerID="56bdcfbd988cda0ec2ab405ec09ae4c62214573c43835cc20f75afd7eb7d813a" exitCode=0 Jul 14 18:08:06.109583 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:06.109163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" event={"ID":"ed28cebc-3944-46b3-92bf-946678bc3030","Type":"ContainerDied","Data":"56bdcfbd988cda0ec2ab405ec09ae4c62214573c43835cc20f75afd7eb7d813a"} Jul 14 18:08:06.109583 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:06.109452 2572 scope.go:117] "RemoveContainer" containerID="56bdcfbd988cda0ec2ab405ec09ae4c62214573c43835cc20f75afd7eb7d813a" Jul 14 18:08:06.110486 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:06.110458 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:08:06.110545 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:06.110524 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:08:06.110587 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:06.110556 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 14 18:08:07.115226 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:07.115191 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" event={"ID":"ed28cebc-3944-46b3-92bf-946678bc3030","Type":"ContainerStarted","Data":"039d550d6418a5f9d6fa888ca48416a8dd349e3ba636c59262968f7e0e69008c"} Jul 14 18:08:08.042740 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:08.042705 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:08:08.043015 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:08.042986 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6t6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:08:08.043182 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:08.043155 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" podUID="ed28cebc-3944-46b3-92bf-946678bc3030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 14 18:08:08.096240 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:08.096210 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67bf4f8764-8nhhp" Jul 14 18:08:08.557307 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:08.557273 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:08.668141 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:08.668102 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:09.041990 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:09.041900 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6t6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:08:09.042273 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:09.042224 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" podUID="ed28cebc-3944-46b3-92bf-946678bc3030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 14 18:08:09.179176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:09.179145 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:09.325686 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:09.325649 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:10.042012 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:10.041966 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6t6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:08:10.042508 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:10.042042 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" podUID="ed28cebc-3944-46b3-92bf-946678bc3030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 14 18:08:11.044873 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:11.044845 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:08:11.045306 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:11.045126 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:08:11.045955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:11.045939 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6t6m6" Jul 14 18:08:12.087680 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:12.087640 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-kvwjb" Jul 14 18:08:12.088025 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:12.087763 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-httf6" Jul 14 18:08:26.395394 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.395351 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq"] Jul 14 18:08:26.398634 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.398590 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.404988 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.404965 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 18:08:26.405657 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.405638 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 14 18:08:26.405897 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.405878 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6f4ng\"" Jul 14 18:08:26.406002 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.405901 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 18:08:26.418582 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.418555 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq"] Jul 14 18:08:26.433895 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.433866 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:26.491279 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.491246 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.491421 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.491314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06eda4d9-d49e-4b98-9ee8-53a660331709-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.491421 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.491361 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmrrw\" (UniqueName: \"kubernetes.io/projected/06eda4d9-d49e-4b98-9ee8-53a660331709-kube-api-access-vmrrw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.491421 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.491388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.491421 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.491409 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.491571 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.491424 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.591956 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.591924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmrrw\" (UniqueName: \"kubernetes.io/projected/06eda4d9-d49e-4b98-9ee8-53a660331709-kube-api-access-vmrrw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.591973 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.591999 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.592018 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.592056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.592090 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06eda4d9-d49e-4b98-9ee8-53a660331709-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592457 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.592437 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592533 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.592503 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.592533 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.592523 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.594317 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.594295 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06eda4d9-d49e-4b98-9ee8-53a660331709-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.594558 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.594540 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06eda4d9-d49e-4b98-9ee8-53a660331709-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.604004 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.603983 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmrrw\" (UniqueName: \"kubernetes.io/projected/06eda4d9-d49e-4b98-9ee8-53a660331709-kube-api-access-vmrrw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq\" (UID: \"06eda4d9-d49e-4b98-9ee8-53a660331709\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.709200 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.709105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:26.848205 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:26.848178 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq"] Jul 14 18:08:26.851016 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:08:26.850988 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06eda4d9_d49e_4b98_9ee8_53a660331709.slice/crio-f52cdd760ee9d8f8e075e3573015e6f635aa490bf5253ec50c9a41d66243f54a WatchSource:0}: Error finding container f52cdd760ee9d8f8e075e3573015e6f635aa490bf5253ec50c9a41d66243f54a: Status 404 returned error can't find the container with id f52cdd760ee9d8f8e075e3573015e6f635aa490bf5253ec50c9a41d66243f54a Jul 14 18:08:27.217854 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:27.217770 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" event={"ID":"06eda4d9-d49e-4b98-9ee8-53a660331709","Type":"ContainerStarted","Data":"f52cdd760ee9d8f8e075e3573015e6f635aa490bf5253ec50c9a41d66243f54a"} Jul 14 18:08:28.223947 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:28.223909 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" event={"ID":"06eda4d9-d49e-4b98-9ee8-53a660331709","Type":"ContainerStarted","Data":"f61b7a8462e0864743672f76c4f06215d2b383ae52612a0874771be032c285ee"} Jul 14 18:08:28.248063 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:28.247967 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" podStartSLOduration=1.110397591 podStartE2EDuration="2.247949467s" totalImagesPullingTime="1.137551876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:28.247094372 +0000 UTC m=+851.109884176" watchObservedRunningTime="2026-07-14 18:08:28.247949467 +0000 UTC m=+851.110739271" Jul 14 18:08:33.183781 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:33.183719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:36.710201 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:36.710165 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:36.710649 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:36.710239 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:36.722254 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:36.722228 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:37.198844 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.198812 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x824q"] Jul 14 18:08:37.202210 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.202192 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.206982 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.206959 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 18:08:37.233112 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.233088 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x824q"] Jul 14 18:08:37.281054 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.281027 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq" Jul 14 18:08:37.287217 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287187 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.287372 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287237 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2llxm\" (UniqueName: \"kubernetes.io/projected/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-kube-api-access-2llxm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.287372 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287253 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:37.287372 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287270 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.287372 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287362 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.287574 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287411 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.287574 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.287444 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.388797 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.388762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2llxm\" (UniqueName: \"kubernetes.io/projected/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-kube-api-access-2llxm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.388977 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.388820 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.388977 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.388874 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.389098 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.388969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.389098 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.389017 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.389209 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.389107 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.389311 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.389290 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.389440 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.389418 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.389501 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.389422 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.391291 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.391264 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.391524 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.391508 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.398570 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.398551 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2llxm\" (UniqueName: \"kubernetes.io/projected/5726ac0f-d7e7-4779-a12b-3d42346bd3f0-kube-api-access-2llxm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x824q\" (UID: \"5726ac0f-d7e7-4779-a12b-3d42346bd3f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.513302 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.513217 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:37.668886 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:37.668762 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x824q"] Jul 14 18:08:37.671734 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:08:37.671708 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5726ac0f_d7e7_4779_a12b_3d42346bd3f0.slice/crio-5b16f606d83cf97c9223c4ec228fd76c3b6f934034fb867c56fc43b636cce03e WatchSource:0}: Error finding container 5b16f606d83cf97c9223c4ec228fd76c3b6f934034fb867c56fc43b636cce03e: Status 404 returned error can't find the container with id 5b16f606d83cf97c9223c4ec228fd76c3b6f934034fb867c56fc43b636cce03e Jul 14 18:08:38.275458 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:38.275358 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" event={"ID":"5726ac0f-d7e7-4779-a12b-3d42346bd3f0","Type":"ContainerStarted","Data":"4d5f688266373a9560dc8729174bede4b3f4878d01e03691a12d543b854bd1d6"} Jul 14 18:08:38.275934 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:38.275459 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" event={"ID":"5726ac0f-d7e7-4779-a12b-3d42346bd3f0","Type":"ContainerStarted","Data":"5b16f606d83cf97c9223c4ec228fd76c3b6f934034fb867c56fc43b636cce03e"} Jul 14 18:08:38.299728 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:38.299684 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" podStartSLOduration=1.11968983 podStartE2EDuration="1.299668241s" totalImagesPullingTime="179.978411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:38.297879082 +0000 UTC m=+861.160668887" watchObservedRunningTime="2026-07-14 18:08:38.299668241 +0000 UTC m=+861.162458046" Jul 14 18:08:44.564780 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:44.564738 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:44.667014 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:44.666974 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:47.514060 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:47.514027 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:47.514477 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:47.514178 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:47.526752 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:47.526729 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:48.336632 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:48.336577 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x824q" Jul 14 18:08:51.188056 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.188024 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z"] Jul 14 18:08:51.190601 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.190581 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.193368 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.193347 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 18:08:51.205808 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.205787 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z"] Jul 14 18:08:51.223302 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.223274 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:51.327591 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.327553 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.327591 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.327589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.327840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.327646 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.327840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.327683 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4w9k\" (UniqueName: \"kubernetes.io/projected/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-kube-api-access-s4w9k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.327840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.327712 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.327840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.327752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.428517 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428481 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.428517 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.428806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428539 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.428806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428577 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4w9k\" (UniqueName: \"kubernetes.io/projected/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-kube-api-access-s4w9k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.428806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428599 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.428806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428668 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.429017 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.428959 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.429017 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.429006 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.429098 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.429030 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.431002 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.430978 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.431252 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.431236 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.437438 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.437417 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4w9k\" (UniqueName: \"kubernetes.io/projected/caa430e5-ded9-49b6-b7d8-0e82d6e84e10-kube-api-access-s4w9k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z\" (UID: \"caa430e5-ded9-49b6-b7d8-0e82d6e84e10\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.501809 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.501715 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:08:51.639492 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:51.639469 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z"] Jul 14 18:08:51.641459 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:08:51.641433 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa430e5_ded9_49b6_b7d8_0e82d6e84e10.slice/crio-3fbffad86197bae0a094136ff3c44dc17cec40489b48405a8a0cd880a1111476 WatchSource:0}: Error finding container 3fbffad86197bae0a094136ff3c44dc17cec40489b48405a8a0cd880a1111476: Status 404 returned error can't find the container with id 3fbffad86197bae0a094136ff3c44dc17cec40489b48405a8a0cd880a1111476 Jul 14 18:08:52.348600 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:52.348564 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" event={"ID":"caa430e5-ded9-49b6-b7d8-0e82d6e84e10","Type":"ContainerStarted","Data":"9fd1822717885b649a6f8b990a8a66408dea5d02fbeb2cd35c156460a38142b2"} Jul 14 18:08:52.349009 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:52.348628 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" event={"ID":"caa430e5-ded9-49b6-b7d8-0e82d6e84e10","Type":"ContainerStarted","Data":"3fbffad86197bae0a094136ff3c44dc17cec40489b48405a8a0cd880a1111476"} Jul 14 18:08:52.368853 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:52.368804 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" podStartSLOduration=1.18200818 podStartE2EDuration="1.368789768s" totalImagesPullingTime="186.781588ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:52.366838105 +0000 UTC m=+875.229627910" watchObservedRunningTime="2026-07-14 18:08:52.368789768 +0000 UTC m=+875.231579571" Jul 14 18:08:58.382319 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.382286 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv"] Jul 14 18:08:58.385225 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.385205 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.391384 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.391359 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 18:08:58.400425 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.400404 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv"] Jul 14 18:08:58.455123 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.455084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:08:58.494961 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.494930 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.495189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.494981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.495189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.495013 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9519b-a724-46d0-a9eb-75b8793c9799-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.495189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.495056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.495189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.495118 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpmfj\" (UniqueName: \"kubernetes.io/projected/ede9519b-a724-46d0-a9eb-75b8793c9799-kube-api-access-fpmfj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.495189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.495141 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.595748 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.595708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.595941 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.595770 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.595941 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.595796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9519b-a724-46d0-a9eb-75b8793c9799-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.595941 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.595815 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.595941 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.595886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpmfj\" (UniqueName: \"kubernetes.io/projected/ede9519b-a724-46d0-a9eb-75b8793c9799-kube-api-access-fpmfj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.595941 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.595909 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.596265 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.596241 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.596311 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.596241 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.596311 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.596290 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.598111 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.598090 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9519b-a724-46d0-a9eb-75b8793c9799-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.598436 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.598420 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9519b-a724-46d0-a9eb-75b8793c9799-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.605758 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.605737 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpmfj\" (UniqueName: \"kubernetes.io/projected/ede9519b-a724-46d0-a9eb-75b8793c9799-kube-api-access-fpmfj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwrxv\" (UID: \"ede9519b-a724-46d0-a9eb-75b8793c9799\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.696289 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.696192 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:08:58.844039 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:58.844017 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv"] Jul 14 18:08:58.846593 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:08:58.846567 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede9519b_a724_46d0_a9eb_75b8793c9799.slice/crio-b34ffbc4dc6d63c4920c48e378254e934bcfe31450b9254d4c58a0c93604d7d4 WatchSource:0}: Error finding container b34ffbc4dc6d63c4920c48e378254e934bcfe31450b9254d4c58a0c93604d7d4: Status 404 returned error can't find the container with id b34ffbc4dc6d63c4920c48e378254e934bcfe31450b9254d4c58a0c93604d7d4 Jul 14 18:08:59.384224 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:59.384187 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" event={"ID":"ede9519b-a724-46d0-a9eb-75b8793c9799","Type":"ContainerStarted","Data":"a087ddc14d2a58d1edb3030ae69104584afea1d1a2576f7f10cc85879453f10b"} Jul 14 18:08:59.384600 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:59.384243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" event={"ID":"ede9519b-a724-46d0-a9eb-75b8793c9799","Type":"ContainerStarted","Data":"b34ffbc4dc6d63c4920c48e378254e934bcfe31450b9254d4c58a0c93604d7d4"} Jul 14 18:08:59.412906 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:08:59.412857 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" podStartSLOduration=1.229478141 podStartE2EDuration="1.412841919s" totalImagesPullingTime="183.363778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:59.41151889 +0000 UTC m=+882.274308693" watchObservedRunningTime="2026-07-14 18:08:59.412841919 +0000 UTC m=+882.275631736" Jul 14 18:09:01.163756 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:01.163718 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:09:01.502137 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:01.502048 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:09:01.502137 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:01.502122 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:09:01.514419 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:01.514397 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:09:02.410896 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:02.410868 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z" Jul 14 18:09:08.697131 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:08.697046 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:09:08.697131 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:08.697110 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:09:08.709945 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:08.709919 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:09:09.449120 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:09.449092 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwrxv" Jul 14 18:09:28.912568 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:28.912532 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:09:29.014266 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:09:29.014223 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:10:26.740507 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:10:26.740470 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:10:30.067305 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:10:30.067264 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:11:40.692640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:40.692588 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:11:40.792062 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:40.792035 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-4tv4t"] Jul 14 18:11:40.792266 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:40.792240 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" podUID="67404d70-da1c-4223-8636-bfa02ecc656b" containerName="manager" containerID="cri-o://654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020" gracePeriod=30 Jul 14 18:11:41.039989 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.039968 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:11:41.201130 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.201103 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67404d70-da1c-4223-8636-bfa02ecc656b-webhook-cert\") pod \"67404d70-da1c-4223-8636-bfa02ecc656b\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " Jul 14 18:11:41.201308 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.201180 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6s8x\" (UniqueName: \"kubernetes.io/projected/67404d70-da1c-4223-8636-bfa02ecc656b-kube-api-access-k6s8x\") pod \"67404d70-da1c-4223-8636-bfa02ecc656b\" (UID: \"67404d70-da1c-4223-8636-bfa02ecc656b\") " Jul 14 18:11:41.203485 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.203451 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67404d70-da1c-4223-8636-bfa02ecc656b-kube-api-access-k6s8x" (OuterVolumeSpecName: "kube-api-access-k6s8x") pod "67404d70-da1c-4223-8636-bfa02ecc656b" (UID: "67404d70-da1c-4223-8636-bfa02ecc656b"). InnerVolumeSpecName "kube-api-access-k6s8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:11:41.203485 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.203459 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67404d70-da1c-4223-8636-bfa02ecc656b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "67404d70-da1c-4223-8636-bfa02ecc656b" (UID: "67404d70-da1c-4223-8636-bfa02ecc656b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 18:11:41.205422 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.205399 2572 generic.go:356] "Generic (PLEG): container finished" podID="67404d70-da1c-4223-8636-bfa02ecc656b" containerID="654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020" exitCode=0 Jul 14 18:11:41.205547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.205443 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" Jul 14 18:11:41.205547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.205483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" event={"ID":"67404d70-da1c-4223-8636-bfa02ecc656b","Type":"ContainerDied","Data":"654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020"} Jul 14 18:11:41.205547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.205545 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bb644f5-4tv4t" event={"ID":"67404d70-da1c-4223-8636-bfa02ecc656b","Type":"ContainerDied","Data":"abbc691936de6141a10ac36ba54f04f74a83145ee71700b6af00aa38bf179ce1"} Jul 14 18:11:41.205750 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.205579 2572 scope.go:117] "RemoveContainer" containerID="654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020" Jul 14 18:11:41.222859 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.222842 2572 scope.go:117] "RemoveContainer" containerID="654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020" Jul 14 18:11:41.223147 ip-10-0-129-115 kubenswrapper[2572]: E0714 18:11:41.223129 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020\": container with ID starting with 654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020 not found: ID does not exist" containerID="654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020" Jul 14 18:11:41.223212 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.223157 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020"} err="failed to get container status \"654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020\": rpc error: code = NotFound desc = could not find container \"654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020\": container with ID starting with 654411d84f2ea3ac8f5bdda36eeb50a2e0e018a5c5f92fbb2e3405c9315d9020 not found: ID does not exist" Jul 14 18:11:41.244577 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.244546 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-4tv4t"] Jul 14 18:11:41.250735 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.250712 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-4tv4t"] Jul 14 18:11:41.301939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.301915 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67404d70-da1c-4223-8636-bfa02ecc656b-webhook-cert\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:11:41.301939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.301939 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6s8x\" (UniqueName: \"kubernetes.io/projected/67404d70-da1c-4223-8636-bfa02ecc656b-kube-api-access-k6s8x\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:11:41.718576 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:41.718548 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67404d70-da1c-4223-8636-bfa02ecc656b" path="/var/lib/kubelet/pods/67404d70-da1c-4223-8636-bfa02ecc656b/volumes" Jul 14 18:11:42.113448 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.113413 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:11:42.148506 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.148471 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-9cwg5"] Jul 14 18:11:42.148947 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.148933 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67404d70-da1c-4223-8636-bfa02ecc656b" containerName="manager" Jul 14 18:11:42.148994 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.148949 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="67404d70-da1c-4223-8636-bfa02ecc656b" containerName="manager" Jul 14 18:11:42.149035 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.149024 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="67404d70-da1c-4223-8636-bfa02ecc656b" containerName="manager" Jul 14 18:11:42.153432 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.153415 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.159820 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.159796 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 18:11:42.159820 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.159813 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bj86r\"" Jul 14 18:11:42.160013 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.159796 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 18:11:42.168014 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.167993 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-9cwg5"] Jul 14 18:11:42.311239 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.311210 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a-webhook-cert\") pod \"maas-controller-5f4bb644f5-9cwg5\" (UID: \"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a\") " pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.311400 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.311351 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8pt2\" (UniqueName: \"kubernetes.io/projected/0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a-kube-api-access-b8pt2\") pod \"maas-controller-5f4bb644f5-9cwg5\" (UID: \"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a\") " pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.411879 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.411795 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8pt2\" (UniqueName: \"kubernetes.io/projected/0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a-kube-api-access-b8pt2\") pod \"maas-controller-5f4bb644f5-9cwg5\" (UID: \"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a\") " pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.411879 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.411839 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a-webhook-cert\") pod \"maas-controller-5f4bb644f5-9cwg5\" (UID: \"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a\") " pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.414274 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.414255 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a-webhook-cert\") pod \"maas-controller-5f4bb644f5-9cwg5\" (UID: \"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a\") " pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.421961 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.421941 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8pt2\" (UniqueName: \"kubernetes.io/projected/0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a-kube-api-access-b8pt2\") pod \"maas-controller-5f4bb644f5-9cwg5\" (UID: \"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a\") " pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.464363 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.464339 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:42.613058 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.613036 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4bb644f5-9cwg5"] Jul 14 18:11:42.614828 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:11:42.614798 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf5a7b4_304d_4d33_aa11_c0a8cf94af9a.slice/crio-bda4c2e91d1871dd2835ff8ba2fb8e2b41593ef7da9aebabfb0d681077f501aa WatchSource:0}: Error finding container bda4c2e91d1871dd2835ff8ba2fb8e2b41593ef7da9aebabfb0d681077f501aa: Status 404 returned error can't find the container with id bda4c2e91d1871dd2835ff8ba2fb8e2b41593ef7da9aebabfb0d681077f501aa Jul 14 18:11:42.616210 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:42.616193 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 18:11:43.218364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:43.218330 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" event={"ID":"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a","Type":"ContainerStarted","Data":"bda4c2e91d1871dd2835ff8ba2fb8e2b41593ef7da9aebabfb0d681077f501aa"} Jul 14 18:11:44.225344 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:44.225305 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" event={"ID":"0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a","Type":"ContainerStarted","Data":"6697b451889f6c53822ba2fc1bdfc81c43d9f60f58f205702c1bd783b1897589"} Jul 14 18:11:44.225754 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:44.225406 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:11:44.256905 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:44.256814 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" podStartSLOduration=0.88770386 podStartE2EDuration="2.256798336s" totalImagesPullingTime="1.369094476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:11:44.256794102 +0000 UTC m=+1047.119583929" watchObservedRunningTime="2026-07-14 18:11:44.256798336 +0000 UTC m=+1047.119588140" Jul 14 18:11:55.233070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:11:55.233041 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f4bb644f5-9cwg5" Jul 14 18:12:12.582155 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:12.582074 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:12.682192 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:12.682160 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:22.108091 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:22.108053 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:26.542818 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:26.542772 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:26.653761 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:26.653719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:26.760142 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:26.760105 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:33.972482 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:33.972444 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:34.116597 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:34.116557 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:34.296019 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:34.295923 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:34.459926 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:34.459886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:41.274368 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:41.274319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:12:41.403224 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:12:41.403173 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:13:42.674405 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:13:42.674325 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:13:44.461038 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:13:44.461006 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:14:08.260367 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:14:08.260319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:14:32.694470 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:14:32.694424 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:18:18.897724 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:18:18.897644 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:18:22.038130 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:18:22.038098 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:19:04.229495 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:19:04.229460 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:19:04.372119 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:19:04.372084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:19:58.194009 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:19:58.193926 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:20:30.847978 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:20:30.847933 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:21:58.940751 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:21:58.940711 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:23:40.870125 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:23:40.870084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:23:48.337303 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:23:48.337266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:23:56.708763 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:23:56.708727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:01.735254 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:01.735217 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:04.727245 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:04.727208 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:04.817533 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:04.817492 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:05.215408 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:05.215373 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:12.220970 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:12.220934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:13.936282 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:13.936249 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:14.116899 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:14.116871 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:14.644604 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:14.644572 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:14.754494 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:14.754458 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:24:16.886868 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:24:16.886812 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:19.460904 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:19.460867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:23.646043 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:23.645999 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:30.291534 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:30.291494 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:30.392518 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:30.392478 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:33.849869 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:33.849832 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:34.217073 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:34.216979 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:34.320958 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:34.320920 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:35.295466 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:35.295429 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:35.788187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:35.788154 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:35.910810 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:35.910775 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:36.301580 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:36.301536 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:36.498321 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:36.498287 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:36.585525 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:36.585435 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:40.423672 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:40.423635 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:40.955525 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:40.955489 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:25:44.058669 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:25:44.058592 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:37.544101 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:37.544044 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:40.445839 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:40.445788 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:40.531342 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:40.531310 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:43.029963 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:43.029932 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:43.125466 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:43.125400 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:45.856318 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:45.856278 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:46.769311 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:46.769283 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:47.690965 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:47.690935 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:49.137196 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:49.137158 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:49.261359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:49.261326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:52.794247 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:52.794213 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:52.855660 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:52.855594 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:53.048864 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:53.048830 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:53.148727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:53.148687 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:54.132567 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:54.132524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:54.244349 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:54.244303 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:54.325845 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:54.325807 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:54.442481 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:54.442388 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:54.684229 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:54.684197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:54.785683 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:54.785571 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:56.871373 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:56.871341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:26:59.230192 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.230149 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t"] Jul 14 18:26:59.233980 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.233957 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:26:59.236642 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.236596 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:26:59.241445 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.241418 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t"] Jul 14 18:26:59.387043 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.387014 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801d0b7f-2ddf-46cb-a5cb-a520af544e19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t\" (UID: \"801d0b7f-2ddf-46cb-a5cb-a520af544e19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:26:59.487890 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.487803 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801d0b7f-2ddf-46cb-a5cb-a520af544e19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t\" (UID: \"801d0b7f-2ddf-46cb-a5cb-a520af544e19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:26:59.488389 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.488364 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801d0b7f-2ddf-46cb-a5cb-a520af544e19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t\" (UID: \"801d0b7f-2ddf-46cb-a5cb-a520af544e19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:26:59.546950 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.546925 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:26:59.692301 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.692273 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t"] Jul 14 18:26:59.694492 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:26:59.694468 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801d0b7f_2ddf_46cb_a5cb_a520af544e19.slice/crio-95e8f202baee61d10725982dc29638cc95a50cdb95e1e757ba4d9b64c4b64529 WatchSource:0}: Error finding container 95e8f202baee61d10725982dc29638cc95a50cdb95e1e757ba4d9b64c4b64529: Status 404 returned error can't find the container with id 95e8f202baee61d10725982dc29638cc95a50cdb95e1e757ba4d9b64c4b64529 Jul 14 18:26:59.696777 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.696756 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 18:26:59.835505 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:26:59.835464 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" event={"ID":"801d0b7f-2ddf-46cb-a5cb-a520af544e19","Type":"ContainerStarted","Data":"95e8f202baee61d10725982dc29638cc95a50cdb95e1e757ba4d9b64c4b64529"} Jul 14 18:27:00.264469 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:00.264388 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:00.364987 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:00.364949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:00.464826 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:00.464791 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:02.852624 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:02.852579 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" event={"ID":"801d0b7f-2ddf-46cb-a5cb-a520af544e19","Type":"ContainerStarted","Data":"9c04835142b8a00e9aa24178be5ab703cacabac044cff45a85b8124de27ea095"} Jul 14 18:27:02.870946 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:02.870859 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" podStartSLOduration=1.082873214 podStartE2EDuration="3.870843679s" totalImagesPullingTime="2.787970465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:27:02.870091024 +0000 UTC m=+1965.732880828" watchObservedRunningTime="2026-07-14 18:27:02.870843679 +0000 UTC m=+1965.733633480" Jul 14 18:27:03.859368 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:03.859326 2572 generic.go:356] "Generic (PLEG): container finished" podID="801d0b7f-2ddf-46cb-a5cb-a520af544e19" containerID="9c04835142b8a00e9aa24178be5ab703cacabac044cff45a85b8124de27ea095" exitCode=0 Jul 14 18:27:03.859758 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:03.859413 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" event={"ID":"801d0b7f-2ddf-46cb-a5cb-a520af544e19","Type":"ContainerDied","Data":"9c04835142b8a00e9aa24178be5ab703cacabac044cff45a85b8124de27ea095"} Jul 14 18:27:05.003227 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.003197 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:27:05.142919 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.142832 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801d0b7f-2ddf-46cb-a5cb-a520af544e19-maas-api-service-ca\") pod \"801d0b7f-2ddf-46cb-a5cb-a520af544e19\" (UID: \"801d0b7f-2ddf-46cb-a5cb-a520af544e19\") " Jul 14 18:27:05.143205 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.143183 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/801d0b7f-2ddf-46cb-a5cb-a520af544e19-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "801d0b7f-2ddf-46cb-a5cb-a520af544e19" (UID: "801d0b7f-2ddf-46cb-a5cb-a520af544e19"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:27:05.244067 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.244035 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801d0b7f-2ddf-46cb-a5cb-a520af544e19-maas-api-service-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:27:05.870800 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.870772 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" Jul 14 18:27:05.870958 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.870773 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t" event={"ID":"801d0b7f-2ddf-46cb-a5cb-a520af544e19","Type":"ContainerDied","Data":"95e8f202baee61d10725982dc29638cc95a50cdb95e1e757ba4d9b64c4b64529"} Jul 14 18:27:05.870958 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:05.870896 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95e8f202baee61d10725982dc29638cc95a50cdb95e1e757ba4d9b64c4b64529" Jul 14 18:27:06.687681 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:06.687638 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:06.785529 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:06.785497 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:09.807832 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:09.807740 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:09.908295 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:09.908268 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:10.029207 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:10.029172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:14.330000 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:14.329962 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:20.573177 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:20.573147 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:20.676238 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:20.676206 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:25.359126 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:25.359089 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:25.667604 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:25.667569 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:26.786359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:26.786315 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:26.887246 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:26.887215 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:26.979146 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:26.979105 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:27.112394 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:27.112353 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:27.569896 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:27.569859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:27.676163 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:27.676129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:31.692590 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:31.692552 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:31.756557 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:31.756522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:27:35.247346 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:27:35.247314 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:41.743351 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:41.743270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:46.353544 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:46.353499 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:52.902041 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:52.901999 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:52.997175 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:52.997141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:56.553537 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:56.553493 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:57.151508 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:57.151458 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:57.252771 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:57.252734 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:57.352252 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:57.352220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:28:58.800402 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:28:58.800364 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:03.055494 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:03.055462 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:05.106980 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:05.106939 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:05.211932 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:05.211895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:05.311159 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:05.311118 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:05.409077 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:05.408993 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:05.816076 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:05.816043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:27.977827 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:27.977794 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:28.040023 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:28.039987 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:31.797190 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:31.797138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:37.996641 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:37.996580 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:38.063071 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:38.063035 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:38.288183 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:38.288098 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:38.390138 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:38.390086 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:42.256164 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:42.256125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:42.311014 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:42.310972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:29:54.127539 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:29:54.127505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:00.313625 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:00.313585 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:00.380518 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:00.380470 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:00.482691 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:00.482658 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:00.580806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:00.580725 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:11.050268 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:11.050180 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:11.850715 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:11.850681 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:17.989292 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:17.989228 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:18.103911 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:18.103877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:18.974899 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:18.974867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:19.082176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:19.082134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:19.389652 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:19.389595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:19.531165 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:19.531134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:21.656228 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:21.656193 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:27.399486 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:27.399446 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:27.467652 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:27.467625 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:28.083317 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:28.083279 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:34.428453 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:34.428409 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:34.499597 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:34.499563 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:34.672466 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:34.672426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:35.024998 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:35.024969 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:35.543711 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:35.543674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:36.430432 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:36.430403 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:38.361519 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:38.361477 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:38.456968 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:38.456937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:38.554229 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:38.554191 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:39.075582 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:39.075545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:39.177306 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:39.177264 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:42.871592 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:42.871556 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:43.529844 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:43.529810 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:43.633960 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:43.633932 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:44.022409 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:44.022355 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:50.089151 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:50.089100 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:30:50.189090 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:30:50.189059 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:00.934321 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:00.934283 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:05.594065 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:05.594028 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:11.938475 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:11.938439 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:12.129601 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:12.129569 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:16.270464 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:16.270423 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:16.504449 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:16.504416 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:16.699403 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:16.699371 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:16.955673 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:16.955566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:17.067670 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:17.067599 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:23.425870 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:23.425841 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:23.506046 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:23.506021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:23.637927 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:23.637898 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:23.777372 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:23.777281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:30.063058 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:30.063021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:30.168543 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:30.168508 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:37.156082 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:37.156047 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:37.475958 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:37.475865 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:37.841767 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:37.841729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:40.808212 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:40.808120 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:40.908677 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:40.908637 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:41.006659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:41.006625 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:41.306315 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:41.306280 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:41.473168 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:41.473135 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:41.574352 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:41.574265 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:45.271355 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:45.271319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:31:54.303169 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:31:54.303132 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:07.582138 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:07.582104 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:07.781916 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:07.781881 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:30.135187 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:30.135150 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:30.264662 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:30.264600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:39.990547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:39.990515 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:44.419156 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:44.419110 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:32:49.444639 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:32:49.444582 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:15.637957 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:15.637868 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:26.461955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:26.461908 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:27.817737 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:27.817699 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:34.350905 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:34.350864 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:34.441037 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:34.441005 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:40.613309 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:40.613272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:40.730886 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:40.730848 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:46.742531 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:46.742493 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:46.996229 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:46.996141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:48.137570 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:48.137531 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:50.294753 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:50.294720 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:50.820125 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:50.820092 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:57.407514 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:57.407476 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:57.503826 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:57.503780 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:33:58.154956 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:33:58.154910 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:05.082055 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:05.082009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:05.194256 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:05.194136 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:05.305588 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:05.305553 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:15.811800 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:15.811765 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:16.225466 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:16.225378 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:37.207693 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:37.207661 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:43.755060 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:43.754975 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:43.861181 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:43.861139 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:44.163487 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:44.163455 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:44.490865 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:44.490789 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:44.597978 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:44.597941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:47.932796 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:47.932756 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:48.067594 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:48.067568 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:48.187325 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:48.187222 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:48.311654 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:48.311603 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:48.740048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:48.739999 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:48.850314 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:48.850284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:52.500550 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:52.500506 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:52.869017 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:52.868983 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:53.352806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:53.352759 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:53.469253 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:53.469207 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:55.970545 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:55.970502 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:57.909767 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:57.909726 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:34:58.008247 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:34:58.008212 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:00.558428 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:00.558387 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:09.690278 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:09.690229 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:09.794675 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:09.794631 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:17.724028 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:17.723982 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:17.825212 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:17.825179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:28.385358 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:28.385321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:35:48.809883 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:35:48.809841 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:10.185924 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:10.185841 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:11.129985 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.129942 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:11.238335 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.238295 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq"] Jul 14 18:36:11.239050 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.239029 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="801d0b7f-2ddf-46cb-a5cb-a520af544e19" containerName="revoke-keys" Jul 14 18:36:11.239101 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.239055 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="801d0b7f-2ddf-46cb-a5cb-a520af544e19" containerName="revoke-keys" Jul 14 18:36:11.239176 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.239165 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="801d0b7f-2ddf-46cb-a5cb-a520af544e19" containerName="revoke-keys" Jul 14 18:36:11.242796 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.242777 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:11.245806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.245784 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:36:11.248885 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.248846 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f552588-312a-43ed-969f-30dc22ce74c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq\" (UID: \"9f552588-312a-43ed-969f-30dc22ce74c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:11.251309 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.251283 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq"] Jul 14 18:36:11.349303 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.349269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f552588-312a-43ed-969f-30dc22ce74c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq\" (UID: \"9f552588-312a-43ed-969f-30dc22ce74c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:11.349933 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.349911 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f552588-312a-43ed-969f-30dc22ce74c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq\" (UID: \"9f552588-312a-43ed-969f-30dc22ce74c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:11.555544 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.555512 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:11.698242 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.697675 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq"] Jul 14 18:36:11.708878 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:36:11.708841 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f552588_312a_43ed_969f_30dc22ce74c8.slice/crio-7d71462e87b51481589b70749997360dc35cbd369b5d17e2cc52440ac46f2aee WatchSource:0}: Error finding container 7d71462e87b51481589b70749997360dc35cbd369b5d17e2cc52440ac46f2aee: Status 404 returned error can't find the container with id 7d71462e87b51481589b70749997360dc35cbd369b5d17e2cc52440ac46f2aee Jul 14 18:36:11.834391 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.834301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:11.934694 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:11.934653 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:12.656116 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:12.656080 2572 generic.go:356] "Generic (PLEG): container finished" podID="9f552588-312a-43ed-969f-30dc22ce74c8" containerID="711a9e155e7bb8efdd8e993ca0c9701122bf476bc94a0633c7360897c8dc04e4" exitCode=0 Jul 14 18:36:12.656548 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:12.656165 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" event={"ID":"9f552588-312a-43ed-969f-30dc22ce74c8","Type":"ContainerDied","Data":"711a9e155e7bb8efdd8e993ca0c9701122bf476bc94a0633c7360897c8dc04e4"} Jul 14 18:36:12.656548 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:12.656214 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" event={"ID":"9f552588-312a-43ed-969f-30dc22ce74c8","Type":"ContainerStarted","Data":"7d71462e87b51481589b70749997360dc35cbd369b5d17e2cc52440ac46f2aee"} Jul 14 18:36:13.805947 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:13.805923 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:13.868838 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:13.868809 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f552588-312a-43ed-969f-30dc22ce74c8-maas-api-service-ca\") pod \"9f552588-312a-43ed-969f-30dc22ce74c8\" (UID: \"9f552588-312a-43ed-969f-30dc22ce74c8\") " Jul 14 18:36:13.869168 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:13.869144 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f552588-312a-43ed-969f-30dc22ce74c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f552588-312a-43ed-969f-30dc22ce74c8" (UID: "9f552588-312a-43ed-969f-30dc22ce74c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:36:13.969581 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:13.969504 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f552588-312a-43ed-969f-30dc22ce74c8-maas-api-service-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:36:14.666721 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:14.666680 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" Jul 14 18:36:14.666721 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:14.666704 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq" event={"ID":"9f552588-312a-43ed-969f-30dc22ce74c8","Type":"ContainerDied","Data":"7d71462e87b51481589b70749997360dc35cbd369b5d17e2cc52440ac46f2aee"} Jul 14 18:36:14.666935 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:14.666754 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d71462e87b51481589b70749997360dc35cbd369b5d17e2cc52440ac46f2aee" Jul 14 18:36:21.777235 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:21.777201 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:26.327578 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:26.327542 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:32.890763 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:32.890727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:32.996417 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:32.996388 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:37.263605 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.263561 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:37.453524 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.453487 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:37.594377 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.594337 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w"] Jul 14 18:36:37.594892 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.594874 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f552588-312a-43ed-969f-30dc22ce74c8" containerName="revoke-keys" Jul 14 18:36:37.594892 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.594893 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f552588-312a-43ed-969f-30dc22ce74c8" containerName="revoke-keys" Jul 14 18:36:37.595001 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.594967 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f552588-312a-43ed-969f-30dc22ce74c8" containerName="revoke-keys" Jul 14 18:36:37.597951 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.597934 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:37.601088 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.601073 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:36:37.609036 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.609018 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w"] Jul 14 18:36:37.687079 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.687046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc631e-980a-421f-b646-1b2f2dec6bb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w\" (UID: \"e1dc631e-980a-421f-b646-1b2f2dec6bb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:37.788105 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.788073 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc631e-980a-421f-b646-1b2f2dec6bb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w\" (UID: \"e1dc631e-980a-421f-b646-1b2f2dec6bb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:37.788727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.788709 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc631e-980a-421f-b646-1b2f2dec6bb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w\" (UID: \"e1dc631e-980a-421f-b646-1b2f2dec6bb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:37.909329 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:37.909247 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:38.044791 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:38.044761 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w"] Jul 14 18:36:38.046221 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:36:38.046194 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dc631e_980a_421f_b646_1b2f2dec6bb8.slice/crio-4c51fd4383f7a16e7bacc0c6f68bfc37670f5930633b3ac11e3ee3c4593da298 WatchSource:0}: Error finding container 4c51fd4383f7a16e7bacc0c6f68bfc37670f5930633b3ac11e3ee3c4593da298: Status 404 returned error can't find the container with id 4c51fd4383f7a16e7bacc0c6f68bfc37670f5930633b3ac11e3ee3c4593da298 Jul 14 18:36:38.237521 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:38.237418 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:38.338829 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:38.338793 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:38.787834 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:38.787803 2572 generic.go:356] "Generic (PLEG): container finished" podID="e1dc631e-980a-421f-b646-1b2f2dec6bb8" containerID="36e120c56d9ebeb73e5c25c7b595a92088afde632047edad6d8385d9708a30ae" exitCode=0 Jul 14 18:36:38.787989 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:38.787878 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" event={"ID":"e1dc631e-980a-421f-b646-1b2f2dec6bb8","Type":"ContainerDied","Data":"36e120c56d9ebeb73e5c25c7b595a92088afde632047edad6d8385d9708a30ae"} Jul 14 18:36:38.787989 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:38.787935 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" event={"ID":"e1dc631e-980a-421f-b646-1b2f2dec6bb8","Type":"ContainerStarted","Data":"4c51fd4383f7a16e7bacc0c6f68bfc37670f5930633b3ac11e3ee3c4593da298"} Jul 14 18:36:39.926508 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:39.926482 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:40.008469 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:40.008443 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc631e-980a-421f-b646-1b2f2dec6bb8-maas-api-service-ca\") pod \"e1dc631e-980a-421f-b646-1b2f2dec6bb8\" (UID: \"e1dc631e-980a-421f-b646-1b2f2dec6bb8\") " Jul 14 18:36:40.008806 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:40.008785 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1dc631e-980a-421f-b646-1b2f2dec6bb8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1dc631e-980a-421f-b646-1b2f2dec6bb8" (UID: "e1dc631e-980a-421f-b646-1b2f2dec6bb8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:36:40.109234 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:40.109206 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1dc631e-980a-421f-b646-1b2f2dec6bb8-maas-api-service-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:36:40.799967 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:40.799939 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" Jul 14 18:36:40.800171 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:40.799972 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w" event={"ID":"e1dc631e-980a-421f-b646-1b2f2dec6bb8","Type":"ContainerDied","Data":"4c51fd4383f7a16e7bacc0c6f68bfc37670f5930633b3ac11e3ee3c4593da298"} Jul 14 18:36:40.800171 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:40.800023 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c51fd4383f7a16e7bacc0c6f68bfc37670f5930633b3ac11e3ee3c4593da298" Jul 14 18:36:48.167269 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:48.167230 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:52.697977 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:52.697940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:59.236414 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:59.236372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:36:59.315948 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:36:59.315918 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:02.009649 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:02.009586 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:03.472659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:03.472600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:05.228925 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:05.228886 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:05.344017 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:05.343972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:07.940782 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:07.940746 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:08.032056 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:08.032016 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:08.608149 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:08.608111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:08.709443 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:08.709403 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:08.977740 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:08.977624 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:09.080423 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:09.080363 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:09.566893 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:09.566845 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:09.666633 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:09.666576 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:12.351288 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:12.351247 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:12.816079 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:12.816043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:13.252315 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:13.252243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:13.426860 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:13.426826 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:16.788880 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:16.788841 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:16.887561 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:16.887529 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:17.020907 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:17.020872 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:17.254663 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:17.254605 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:17.488727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:17.488673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:17.589785 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:17.589754 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:21.290732 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:21.290696 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:29.387136 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.387090 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx"] Jul 14 18:37:29.387640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.387522 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dc631e-980a-421f-b646-1b2f2dec6bb8" containerName="revoke-keys" Jul 14 18:37:29.387640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.387533 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dc631e-980a-421f-b646-1b2f2dec6bb8" containerName="revoke-keys" Jul 14 18:37:29.387640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.387593 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dc631e-980a-421f-b646-1b2f2dec6bb8" containerName="revoke-keys" Jul 14 18:37:29.390646 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.390624 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:29.393510 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.393488 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:37:29.399301 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.399281 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx"] Jul 14 18:37:29.474053 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.474017 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx\" (UID: \"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:29.575580 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.575544 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx\" (UID: \"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:29.576189 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.576170 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx\" (UID: \"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:29.702795 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.702708 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:29.839411 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.839387 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx"] Jul 14 18:37:29.841686 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:37:29.841658 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01f87f1_3cb2_4a9f_a2a3_a9889a0b706a.slice/crio-abfc3e36db6f3a19225c648cbe68ab64399cbe9cbc3f8b8a602578552fe904f2 WatchSource:0}: Error finding container abfc3e36db6f3a19225c648cbe68ab64399cbe9cbc3f8b8a602578552fe904f2: Status 404 returned error can't find the container with id abfc3e36db6f3a19225c648cbe68ab64399cbe9cbc3f8b8a602578552fe904f2 Jul 14 18:37:29.916814 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:29.916786 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:30.023974 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:30.023940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:30.053624 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:30.053582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" event={"ID":"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a","Type":"ContainerStarted","Data":"abfc3e36db6f3a19225c648cbe68ab64399cbe9cbc3f8b8a602578552fe904f2"} Jul 14 18:37:31.061400 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:31.061366 2572 generic.go:356] "Generic (PLEG): container finished" podID="b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a" containerID="c75bb078bb5c27f81f56fa600cc48dd4fe2e1c31b5bcd1e76e8838e6b0f42918" exitCode=0 Jul 14 18:37:31.061802 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:31.061447 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" event={"ID":"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a","Type":"ContainerDied","Data":"c75bb078bb5c27f81f56fa600cc48dd4fe2e1c31b5bcd1e76e8838e6b0f42918"} Jul 14 18:37:32.196868 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:32.196837 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:32.303640 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:32.303583 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a-maas-api-service-ca\") pod \"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a\" (UID: \"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a\") " Jul 14 18:37:32.304004 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:32.303976 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a" (UID: "b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:37:32.404747 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:32.404666 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a-maas-api-service-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:37:33.080214 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:33.080186 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" Jul 14 18:37:33.080380 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:33.080183 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx" event={"ID":"b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a","Type":"ContainerDied","Data":"abfc3e36db6f3a19225c648cbe68ab64399cbe9cbc3f8b8a602578552fe904f2"} Jul 14 18:37:33.080380 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:33.080333 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abfc3e36db6f3a19225c648cbe68ab64399cbe9cbc3f8b8a602578552fe904f2" Jul 14 18:37:38.341188 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:38.341141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:38.442023 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:38.441983 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:40.045376 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:40.045339 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:40.151228 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:40.151190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:40.391286 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:40.391248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:44.489767 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:44.489727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:50.661595 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:50.661558 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:50.761278 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:50.761249 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:55.189571 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.189532 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:55.436735 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.436700 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:55.542603 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.542530 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4"] Jul 14 18:37:55.543030 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.543014 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a" containerName="revoke-keys" Jul 14 18:37:55.543083 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.543031 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a" containerName="revoke-keys" Jul 14 18:37:55.543123 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.543106 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a" containerName="revoke-keys" Jul 14 18:37:55.546200 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.546178 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:37:55.548758 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.548737 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:37:55.554499 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.554475 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4"] Jul 14 18:37:55.627116 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.627084 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f43aa09-e9cb-4905-b5e6-9498194e738f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b418a4-svfr4\" (UID: \"3f43aa09-e9cb-4905-b5e6-9498194e738f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:37:55.727542 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.727513 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f43aa09-e9cb-4905-b5e6-9498194e738f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b418a4-svfr4\" (UID: \"3f43aa09-e9cb-4905-b5e6-9498194e738f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:37:55.728175 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.728150 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f43aa09-e9cb-4905-b5e6-9498194e738f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b418a4-svfr4\" (UID: \"3f43aa09-e9cb-4905-b5e6-9498194e738f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:37:55.857734 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:55.857706 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:37:56.003134 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:56.002959 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4"] Jul 14 18:37:56.005444 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:37:56.005421 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f43aa09_e9cb_4905_b5e6_9498194e738f.slice/crio-d9a0edf2411860f9dbeb577a18a5e594251e8db006a345c80e697259ffbfee32 WatchSource:0}: Error finding container d9a0edf2411860f9dbeb577a18a5e594251e8db006a345c80e697259ffbfee32: Status 404 returned error can't find the container with id d9a0edf2411860f9dbeb577a18a5e594251e8db006a345c80e697259ffbfee32 Jul 14 18:37:56.202317 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:56.202217 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" event={"ID":"3f43aa09-e9cb-4905-b5e6-9498194e738f","Type":"ContainerStarted","Data":"d9a0edf2411860f9dbeb577a18a5e594251e8db006a345c80e697259ffbfee32"} Jul 14 18:37:56.476220 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:56.476142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:56.574451 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:56.574405 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:37:57.209766 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:57.209734 2572 generic.go:356] "Generic (PLEG): container finished" podID="3f43aa09-e9cb-4905-b5e6-9498194e738f" containerID="d907bac5bf38b914acca181ff4c6c8195ccb3bb99034d96269799851504149ea" exitCode=0 Jul 14 18:37:57.210213 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:57.209812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" event={"ID":"3f43aa09-e9cb-4905-b5e6-9498194e738f","Type":"ContainerDied","Data":"d907bac5bf38b914acca181ff4c6c8195ccb3bb99034d96269799851504149ea"} Jul 14 18:37:58.356463 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:58.356439 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:37:58.452620 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:58.452583 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f43aa09-e9cb-4905-b5e6-9498194e738f-maas-api-service-ca\") pod \"3f43aa09-e9cb-4905-b5e6-9498194e738f\" (UID: \"3f43aa09-e9cb-4905-b5e6-9498194e738f\") " Jul 14 18:37:58.452970 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:58.452948 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f43aa09-e9cb-4905-b5e6-9498194e738f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f43aa09-e9cb-4905-b5e6-9498194e738f" (UID: "3f43aa09-e9cb-4905-b5e6-9498194e738f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:37:58.553884 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:58.553853 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f43aa09-e9cb-4905-b5e6-9498194e738f-maas-api-service-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:37:59.221685 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:59.221645 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" event={"ID":"3f43aa09-e9cb-4905-b5e6-9498194e738f","Type":"ContainerDied","Data":"d9a0edf2411860f9dbeb577a18a5e594251e8db006a345c80e697259ffbfee32"} Jul 14 18:37:59.221865 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:59.221715 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9a0edf2411860f9dbeb577a18a5e594251e8db006a345c80e697259ffbfee32" Jul 14 18:37:59.221865 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:37:59.221661 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b418a4-svfr4" Jul 14 18:38:06.107526 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:06.107493 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:10.714905 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:10.714869 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:17.122684 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:17.122646 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:17.230655 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:17.230594 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:21.350372 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:21.350339 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:21.589891 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:21.589858 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:21.963316 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:21.963288 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:22.267822 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:22.267733 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:22.493049 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:22.493019 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:25.195346 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:25.195299 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:25.336405 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:25.336372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:25.504798 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:25.504695 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:26.049125 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:26.049077 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:26.147097 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:26.147063 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:26.292220 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:26.292179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:29.666516 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:29.666479 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:30.027167 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:30.027065 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:30.348356 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:30.348317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:30.445327 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:30.445289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:33.218472 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:33.218436 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:33.388396 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:33.388351 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:33.550127 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:33.550085 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:33.935024 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:33.934924 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:34.034944 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:34.034900 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:36.485212 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:36.485172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:37.694553 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:37.694521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:43.846369 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:43.846331 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:43.947261 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:43.947225 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:45.156527 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:45.156485 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:46.866039 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:46.866008 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:46.960768 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:46.960731 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:49.110235 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:49.110199 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:54.862728 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:54.862693 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:38:57.539493 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:38:57.539453 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:39:08.236965 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:39:08.236927 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:40:45.064350 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:40:45.064254 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:40:45.137188 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:40:45.137149 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:41:03.179354 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:41:03.179315 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:41:04.110470 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:41:04.110424 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:41:08.004405 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:41:08.004367 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:41:33.708017 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:41:33.707975 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:41:48.948412 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:41:48.948370 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:09.547990 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:09.547894 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:09.646203 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:09.646172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:20.840158 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:20.840122 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:21.792184 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:21.792130 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:36.472386 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:36.472352 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:49.156143 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:49.156102 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:42:49.252726 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:42:49.252691 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:43:28.622222 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:43:28.622129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:43:47.747248 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:43:47.747169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:43:47.839023 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:43:47.838989 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:43:55.701088 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:43:55.701052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:43:55.802397 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:43:55.802364 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:43:59.018866 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:43:59.018826 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:44:46.630384 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:46.630346 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:44:47.514310 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:47.514268 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:44:48.475627 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.475589 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7546-140320127258416"] Jul 14 18:44:48.476144 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.476128 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f43aa09-e9cb-4905-b5e6-9498194e738f" containerName="revoke-keys" Jul 14 18:44:48.476144 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.476146 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f43aa09-e9cb-4905-b5e6-9498194e738f" containerName="revoke-keys" Jul 14 18:44:48.476255 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.476217 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f43aa09-e9cb-4905-b5e6-9498194e738f" containerName="revoke-keys" Jul 14 18:44:48.479493 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.479475 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:48.481974 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.481953 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-lddj2\"" Jul 14 18:44:48.486642 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.486595 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127258416"] Jul 14 18:44:48.615168 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.615133 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v57zv\" (UniqueName: \"kubernetes.io/projected/978a82ad-9261-4060-92bb-e9d4184dde81-kube-api-access-v57zv\") pod \"test-curl-7546-140320127258416\" (UID: \"978a82ad-9261-4060-92bb-e9d4184dde81\") " pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:48.715632 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.715588 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v57zv\" (UniqueName: \"kubernetes.io/projected/978a82ad-9261-4060-92bb-e9d4184dde81-kube-api-access-v57zv\") pod \"test-curl-7546-140320127258416\" (UID: \"978a82ad-9261-4060-92bb-e9d4184dde81\") " pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:48.726767 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.726705 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v57zv\" (UniqueName: \"kubernetes.io/projected/978a82ad-9261-4060-92bb-e9d4184dde81-kube-api-access-v57zv\") pod \"test-curl-7546-140320127258416\" (UID: \"978a82ad-9261-4060-92bb-e9d4184dde81\") " pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:48.790083 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.790056 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:48.930149 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.930128 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127258416"] Jul 14 18:44:48.935818 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:48.935794 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 18:44:49.402930 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:49.402894 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127258416" event={"ID":"978a82ad-9261-4060-92bb-e9d4184dde81","Type":"ContainerStarted","Data":"b2cd8ee72e2d50195ad9b00bc95000f4673eae70de3d85a560809c3cf9ac1071"} Jul 14 18:44:50.409703 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:50.409672 2572 generic.go:356] "Generic (PLEG): container finished" podID="978a82ad-9261-4060-92bb-e9d4184dde81" containerID="f810b51e9c2df0173662b0582a0ac6a1b403f480896e34d13aa21e15ef90b7ac" exitCode=0 Jul 14 18:44:50.410070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:50.409727 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127258416" event={"ID":"978a82ad-9261-4060-92bb-e9d4184dde81","Type":"ContainerDied","Data":"f810b51e9c2df0173662b0582a0ac6a1b403f480896e34d13aa21e15ef90b7ac"} Jul 14 18:44:51.566144 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.566120 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:51.587923 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.587894 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7546-140320127258416_978a82ad-9261-4060-92bb-e9d4184dde81/test-curl-7546-140320127258416/0.log" Jul 14 18:44:51.616255 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.616217 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7546-140320127258416"] Jul 14 18:44:51.620698 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.620669 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7546-140320127258416"] Jul 14 18:44:51.746175 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.746087 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v57zv\" (UniqueName: \"kubernetes.io/projected/978a82ad-9261-4060-92bb-e9d4184dde81-kube-api-access-v57zv\") pod \"978a82ad-9261-4060-92bb-e9d4184dde81\" (UID: \"978a82ad-9261-4060-92bb-e9d4184dde81\") " Jul 14 18:44:51.748314 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.748286 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978a82ad-9261-4060-92bb-e9d4184dde81-kube-api-access-v57zv" (OuterVolumeSpecName: "kube-api-access-v57zv") pod "978a82ad-9261-4060-92bb-e9d4184dde81" (UID: "978a82ad-9261-4060-92bb-e9d4184dde81"). InnerVolumeSpecName "kube-api-access-v57zv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:44:51.847723 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:51.847687 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v57zv\" (UniqueName: \"kubernetes.io/projected/978a82ad-9261-4060-92bb-e9d4184dde81-kube-api-access-v57zv\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:44:52.422762 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:52.422730 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127258416" Jul 14 18:44:52.422762 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:52.422750 2572 scope.go:117] "RemoveContainer" containerID="f810b51e9c2df0173662b0582a0ac6a1b403f480896e34d13aa21e15ef90b7ac" Jul 14 18:44:53.719395 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:53.719359 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="978a82ad-9261-4060-92bb-e9d4184dde81" path="/var/lib/kubelet/pods/978a82ad-9261-4060-92bb-e9d4184dde81/volumes" Jul 14 18:44:54.887162 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.887131 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"] Jul 14 18:44:54.887561 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.887538 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="978a82ad-9261-4060-92bb-e9d4184dde81" containerName="test-curl-7546-140320127258416" Jul 14 18:44:54.887561 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.887553 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="978a82ad-9261-4060-92bb-e9d4184dde81" containerName="test-curl-7546-140320127258416" Jul 14 18:44:54.887736 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.887651 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="978a82ad-9261-4060-92bb-e9d4184dde81" containerName="test-curl-7546-140320127258416" Jul 14 18:44:54.891970 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.891951 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:54.895519 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.895498 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-lddj2\"" Jul 14 18:44:54.900046 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:54.900023 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"] Jul 14 18:44:55.078364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:55.078321 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qctq\" (UniqueName: \"kubernetes.io/projected/631d0aa2-b08e-450e-a146-def25a4197ca-kube-api-access-4qctq\") pod \"test-curl-7546-140320127037488\" (UID: \"631d0aa2-b08e-450e-a146-def25a4197ca\") " pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:55.179938 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:55.179853 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qctq\" (UniqueName: \"kubernetes.io/projected/631d0aa2-b08e-450e-a146-def25a4197ca-kube-api-access-4qctq\") pod \"test-curl-7546-140320127037488\" (UID: \"631d0aa2-b08e-450e-a146-def25a4197ca\") " pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:55.189376 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:55.189345 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qctq\" (UniqueName: \"kubernetes.io/projected/631d0aa2-b08e-450e-a146-def25a4197ca-kube-api-access-4qctq\") pod \"test-curl-7546-140320127037488\" (UID: \"631d0aa2-b08e-450e-a146-def25a4197ca\") " pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:55.202219 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:55.202194 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:55.342897 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:55.342840 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"] Jul 14 18:44:55.345335 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:44:55.345309 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod631d0aa2_b08e_450e_a146_def25a4197ca.slice/crio-eedd4630b7520b61ab9e2e6da13aa0b7c0a4c7a82ad86781e05ff0dfa87bba90 WatchSource:0}: Error finding container eedd4630b7520b61ab9e2e6da13aa0b7c0a4c7a82ad86781e05ff0dfa87bba90: Status 404 returned error can't find the container with id eedd4630b7520b61ab9e2e6da13aa0b7c0a4c7a82ad86781e05ff0dfa87bba90 Jul 14 18:44:55.446484 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:55.446396 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127037488" event={"ID":"631d0aa2-b08e-450e-a146-def25a4197ca","Type":"ContainerStarted","Data":"eedd4630b7520b61ab9e2e6da13aa0b7c0a4c7a82ad86781e05ff0dfa87bba90"} Jul 14 18:44:56.452783 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:56.452691 2572 generic.go:356] "Generic (PLEG): container finished" podID="631d0aa2-b08e-450e-a146-def25a4197ca" containerID="f9e2d15164c2f0c5a022e840bac49a1795c250b75e4dde51aacfb73fb869481d" exitCode=0 Jul 14 18:44:56.452783 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:56.452746 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127037488" event={"ID":"631d0aa2-b08e-450e-a146-def25a4197ca","Type":"ContainerDied","Data":"f9e2d15164c2f0c5a022e840bac49a1795c250b75e4dde51aacfb73fb869481d"} Jul 14 18:44:57.602793 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.602766 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:57.638471 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.638445 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7546-140320127037488_631d0aa2-b08e-450e-a146-def25a4197ca/test-curl-7546-140320127037488/0.log" Jul 14 18:44:57.672412 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.672380 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"] Jul 14 18:44:57.677792 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.677763 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"] Jul 14 18:44:57.699338 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.699307 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qctq\" (UniqueName: \"kubernetes.io/projected/631d0aa2-b08e-450e-a146-def25a4197ca-kube-api-access-4qctq\") pod \"631d0aa2-b08e-450e-a146-def25a4197ca\" (UID: \"631d0aa2-b08e-450e-a146-def25a4197ca\") " Jul 14 18:44:57.701450 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.701423 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631d0aa2-b08e-450e-a146-def25a4197ca-kube-api-access-4qctq" (OuterVolumeSpecName: "kube-api-access-4qctq") pod "631d0aa2-b08e-450e-a146-def25a4197ca" (UID: "631d0aa2-b08e-450e-a146-def25a4197ca"). InnerVolumeSpecName "kube-api-access-4qctq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:44:57.725470 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.725394 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="631d0aa2-b08e-450e-a146-def25a4197ca" path="/var/lib/kubelet/pods/631d0aa2-b08e-450e-a146-def25a4197ca/volumes" Jul 14 18:44:57.800367 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:57.800326 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qctq\" (UniqueName: \"kubernetes.io/projected/631d0aa2-b08e-450e-a146-def25a4197ca-kube-api-access-4qctq\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:44:58.465051 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:58.465007 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488" Jul 14 18:44:58.465258 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:44:58.465011 2572 scope.go:117] "RemoveContainer" containerID="f9e2d15164c2f0c5a022e840bac49a1795c250b75e4dde51aacfb73fb869481d" Jul 14 18:45:00.149870 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.149834 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9"] Jul 14 18:45:00.150286 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.150270 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631d0aa2-b08e-450e-a146-def25a4197ca" containerName="test-curl-7546-140320127037488" Jul 14 18:45:00.150332 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.150288 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="631d0aa2-b08e-450e-a146-def25a4197ca" containerName="test-curl-7546-140320127037488" Jul 14 18:45:00.150367 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.150353 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="631d0aa2-b08e-450e-a146-def25a4197ca" containerName="test-curl-7546-140320127037488" Jul 14 18:45:00.154910 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.154885 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf"] Jul 14 18:45:00.155057 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.154997 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" Jul 14 18:45:00.157680 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.157660 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:45:00.158253 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.158236 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk"] Jul 14 18:45:00.158359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.158340 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" Jul 14 18:45:00.162048 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.162026 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9"] Jul 14 18:45:00.162169 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.162137 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" Jul 14 18:45:00.165659 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.165637 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf"] Jul 14 18:45:00.168653 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.168598 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" Jul 14 18:45:00.177770 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.177753 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" Jul 14 18:45:00.181495 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.181476 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk"] Jul 14 18:45:00.224286 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.224258 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" Jul 14 18:45:00.343011 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.342976 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9"] Jul 14 18:45:00.347270 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:45:00.347217 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf964a77e_ea1b_44cc_89f0_114b855ca682.slice/crio-51aacc4eb8cbee61e225022c8d7b3f7fd80d0810bec5d2bd0411b65cd9346a27 WatchSource:0}: Error finding container 51aacc4eb8cbee61e225022c8d7b3f7fd80d0810bec5d2bd0411b65cd9346a27: Status 404 returned error can't find the container with id 51aacc4eb8cbee61e225022c8d7b3f7fd80d0810bec5d2bd0411b65cd9346a27 Jul 14 18:45:00.376594 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.376564 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf"] Jul 14 18:45:00.378807 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:45:00.378780 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca265d5f_7d77_4114_8f1d_be58000a36c6.slice/crio-09da104f2eed3377aeab11340bbc31e84cf05814f5c2e894454be599d76401cc WatchSource:0}: Error finding container 09da104f2eed3377aeab11340bbc31e84cf05814f5c2e894454be599d76401cc: Status 404 returned error can't find the container with id 09da104f2eed3377aeab11340bbc31e84cf05814f5c2e894454be599d76401cc Jul 14 18:45:00.406230 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.406200 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk"] Jul 14 18:45:00.412773 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:45:00.412747 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff17266_617b_42f8_b2eb_ed54bfdd4a58.slice/crio-9d4edf3b6ce1cd35f5b67663fc97a5aa196ba2ccd09dd19434abbdffed0f5abc WatchSource:0}: Error finding container 9d4edf3b6ce1cd35f5b67663fc97a5aa196ba2ccd09dd19434abbdffed0f5abc: Status 404 returned error can't find the container with id 9d4edf3b6ce1cd35f5b67663fc97a5aa196ba2ccd09dd19434abbdffed0f5abc Jul 14 18:45:00.477786 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.477750 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" event={"ID":"bff17266-617b-42f8-b2eb-ed54bfdd4a58","Type":"ContainerStarted","Data":"9d4edf3b6ce1cd35f5b67663fc97a5aa196ba2ccd09dd19434abbdffed0f5abc"} Jul 14 18:45:00.478958 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.478932 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" event={"ID":"f964a77e-ea1b-44cc-89f0-114b855ca682","Type":"ContainerStarted","Data":"51aacc4eb8cbee61e225022c8d7b3f7fd80d0810bec5d2bd0411b65cd9346a27"} Jul 14 18:45:00.480072 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:00.480051 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" event={"ID":"ca265d5f-7d77-4114-8f1d-be58000a36c6","Type":"ContainerStarted","Data":"09da104f2eed3377aeab11340bbc31e84cf05814f5c2e894454be599d76401cc"} Jul 14 18:45:01.486593 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:01.486560 2572 generic.go:356] "Generic (PLEG): container finished" podID="ca265d5f-7d77-4114-8f1d-be58000a36c6" containerID="5a0e78c893678412fb1889275b69313cb8bf0475eb6ce64f1499821a2d437610" exitCode=0 Jul 14 18:45:01.487042 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:01.486651 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" event={"ID":"ca265d5f-7d77-4114-8f1d-be58000a36c6","Type":"ContainerDied","Data":"5a0e78c893678412fb1889275b69313cb8bf0475eb6ce64f1499821a2d437610"} Jul 14 18:45:01.488331 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:01.488308 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" event={"ID":"bff17266-617b-42f8-b2eb-ed54bfdd4a58","Type":"ContainerStarted","Data":"b7baa6b9bcf0f3e1cbd4e7495757076bc0d909e3ba7875ded1ee859aabb6155d"} Jul 14 18:45:01.489718 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:01.489696 2572 generic.go:356] "Generic (PLEG): container finished" podID="f964a77e-ea1b-44cc-89f0-114b855ca682" containerID="1470bd7d1aea907ca2ea05bbf498256d3cfc9c2e26cf592a7f81504714c7626c" exitCode=0 Jul 14 18:45:01.489836 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:01.489747 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" event={"ID":"f964a77e-ea1b-44cc-89f0-114b855ca682","Type":"ContainerDied","Data":"1470bd7d1aea907ca2ea05bbf498256d3cfc9c2e26cf592a7f81504714c7626c"} Jul 14 18:45:01.521136 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:01.521093 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" podStartSLOduration=1.521080032 podStartE2EDuration="1.521080032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:45:01.519051869 +0000 UTC m=+3044.381841674" watchObservedRunningTime="2026-07-14 18:45:01.521080032 +0000 UTC m=+3044.383869814" Jul 14 18:45:02.497393 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:02.497360 2572 generic.go:356] "Generic (PLEG): container finished" podID="bff17266-617b-42f8-b2eb-ed54bfdd4a58" containerID="b7baa6b9bcf0f3e1cbd4e7495757076bc0d909e3ba7875ded1ee859aabb6155d" exitCode=0 Jul 14 18:45:02.497809 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:02.497444 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" event={"ID":"bff17266-617b-42f8-b2eb-ed54bfdd4a58","Type":"ContainerDied","Data":"b7baa6b9bcf0f3e1cbd4e7495757076bc0d909e3ba7875ded1ee859aabb6155d"} Jul 14 18:45:02.660141 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:02.660107 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" Jul 14 18:45:02.663134 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:02.663116 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" Jul 14 18:45:03.503776 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.503747 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" Jul 14 18:45:03.503776 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.503762 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734245-9btn9" event={"ID":"f964a77e-ea1b-44cc-89f0-114b855ca682","Type":"ContainerDied","Data":"51aacc4eb8cbee61e225022c8d7b3f7fd80d0810bec5d2bd0411b65cd9346a27"} Jul 14 18:45:03.504248 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.503812 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51aacc4eb8cbee61e225022c8d7b3f7fd80d0810bec5d2bd0411b65cd9346a27" Jul 14 18:45:03.505287 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.505266 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" event={"ID":"ca265d5f-7d77-4114-8f1d-be58000a36c6","Type":"ContainerDied","Data":"09da104f2eed3377aeab11340bbc31e84cf05814f5c2e894454be599d76401cc"} Jul 14 18:45:03.505358 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.505315 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09da104f2eed3377aeab11340bbc31e84cf05814f5c2e894454be599d76401cc" Jul 14 18:45:03.505358 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.505341 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf" Jul 14 18:45:03.643493 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:03.643469 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" Jul 14 18:45:04.512950 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:04.512912 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" event={"ID":"bff17266-617b-42f8-b2eb-ed54bfdd4a58","Type":"ContainerDied","Data":"9d4edf3b6ce1cd35f5b67663fc97a5aa196ba2ccd09dd19434abbdffed0f5abc"} Jul 14 18:45:04.512950 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:04.512939 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk" Jul 14 18:45:04.513416 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:04.512970 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d4edf3b6ce1cd35f5b67663fc97a5aa196ba2ccd09dd19434abbdffed0f5abc" Jul 14 18:45:06.961655 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.961602 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl"] Jul 14 18:45:06.962312 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962291 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff17266-617b-42f8-b2eb-ed54bfdd4a58" containerName="cleanup" Jul 14 18:45:06.962359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962316 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff17266-617b-42f8-b2eb-ed54bfdd4a58" containerName="cleanup" Jul 14 18:45:06.962359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962344 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca265d5f-7d77-4114-8f1d-be58000a36c6" containerName="cleanup" Jul 14 18:45:06.962359 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962353 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca265d5f-7d77-4114-8f1d-be58000a36c6" containerName="cleanup" Jul 14 18:45:06.962454 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962364 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f964a77e-ea1b-44cc-89f0-114b855ca682" containerName="cleanup" Jul 14 18:45:06.962454 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962373 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f964a77e-ea1b-44cc-89f0-114b855ca682" containerName="cleanup" Jul 14 18:45:06.962511 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962479 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff17266-617b-42f8-b2eb-ed54bfdd4a58" containerName="cleanup" Jul 14 18:45:06.962511 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962490 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f964a77e-ea1b-44cc-89f0-114b855ca682" containerName="cleanup" Jul 14 18:45:06.962511 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.962502 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca265d5f-7d77-4114-8f1d-be58000a36c6" containerName="cleanup" Jul 14 18:45:06.966089 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.966071 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:06.968842 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.968822 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\"" Jul 14 18:45:06.973025 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:06.972955 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl"] Jul 14 18:45:07.081807 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.081772 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66851c07-3714-4848-8b46-e2c986092b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl\" (UID: \"66851c07-3714-4848-8b46-e2c986092b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:07.183207 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.183172 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66851c07-3714-4848-8b46-e2c986092b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl\" (UID: \"66851c07-3714-4848-8b46-e2c986092b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:07.183872 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.183852 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66851c07-3714-4848-8b46-e2c986092b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl\" (UID: \"66851c07-3714-4848-8b46-e2c986092b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:07.278846 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.278761 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:07.420408 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.420374 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl"] Jul 14 18:45:07.422043 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:45:07.422012 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66851c07_3714_4848_8b46_e2c986092b63.slice/crio-6d6b6520f5de30ac0134ca413e7bba2d4de223e3dbdaef4e7ccf4adabd3bf4bd WatchSource:0}: Error finding container 6d6b6520f5de30ac0134ca413e7bba2d4de223e3dbdaef4e7ccf4adabd3bf4bd: Status 404 returned error can't find the container with id 6d6b6520f5de30ac0134ca413e7bba2d4de223e3dbdaef4e7ccf4adabd3bf4bd Jul 14 18:45:07.535049 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.534947 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" event={"ID":"66851c07-3714-4848-8b46-e2c986092b63","Type":"ContainerStarted","Data":"6d6b6520f5de30ac0134ca413e7bba2d4de223e3dbdaef4e7ccf4adabd3bf4bd"} Jul 14 18:45:07.573065 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.573034 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:07.672179 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:07.672146 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:08.541355 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:08.541270 2572 generic.go:356] "Generic (PLEG): container finished" podID="66851c07-3714-4848-8b46-e2c986092b63" containerID="878c46842dabb60c2c537241858a37c9032d300a0fc1a5ee75d914bb0d631992" exitCode=0 Jul 14 18:45:08.541715 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:08.541361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" event={"ID":"66851c07-3714-4848-8b46-e2c986092b63","Type":"ContainerDied","Data":"878c46842dabb60c2c537241858a37c9032d300a0fc1a5ee75d914bb0d631992"} Jul 14 18:45:09.685443 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:09.685416 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:09.806840 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:09.806814 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66851c07-3714-4848-8b46-e2c986092b63-maas-api-service-ca\") pod \"66851c07-3714-4848-8b46-e2c986092b63\" (UID: \"66851c07-3714-4848-8b46-e2c986092b63\") " Jul 14 18:45:09.807209 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:09.807182 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66851c07-3714-4848-8b46-e2c986092b63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66851c07-3714-4848-8b46-e2c986092b63" (UID: "66851c07-3714-4848-8b46-e2c986092b63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:45:09.908135 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:09.908099 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66851c07-3714-4848-8b46-e2c986092b63-maas-api-service-ca\") on node \"ip-10-0-129-115.ec2.internal\" DevicePath \"\"" Jul 14 18:45:10.553591 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:10.553559 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" Jul 14 18:45:10.553789 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:10.553592 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl" event={"ID":"66851c07-3714-4848-8b46-e2c986092b63","Type":"ContainerDied","Data":"6d6b6520f5de30ac0134ca413e7bba2d4de223e3dbdaef4e7ccf4adabd3bf4bd"} Jul 14 18:45:10.553789 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:10.553663 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d6b6520f5de30ac0134ca413e7bba2d4de223e3dbdaef4e7ccf4adabd3bf4bd" Jul 14 18:45:17.660791 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:17.660760 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:17.667987 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:17.667957 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf"] Jul 14 18:45:17.671454 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:17.671427 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e7ccb-29734245-9t6tf"] Jul 14 18:45:17.721188 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:17.721146 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca265d5f-7d77-4114-8f1d-be58000a36c6" path="/var/lib/kubelet/pods/ca265d5f-7d77-4114-8f1d-be58000a36c6/volumes" Jul 14 18:45:22.202593 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:22.202557 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:26.781355 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:26.781312 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:26.888413 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:26.888372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:29.385537 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:29.385485 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:29.493288 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:29.493248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:32.882474 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:32.882412 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:33.096764 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:33.096722 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:34.247975 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:34.247942 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:34.355478 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:34.355445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:43.887055 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:43.887012 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:43.889432 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:43.889405 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk"] Jul 14 18:45:43.894534 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:43.894512 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b860a5-29734245-kp4sk"] Jul 14 18:45:45.718937 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:45.718904 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bff17266-617b-42f8-b2eb-ed54bfdd4a58" path="/var/lib/kubelet/pods/bff17266-617b-42f8-b2eb-ed54bfdd4a58/volumes" Jul 14 18:45:48.378871 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:48.378826 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:55.306351 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:55.306318 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:55.405429 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:55.405399 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:59.177142 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:59.177105 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:45:59.394429 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:45:59.394393 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:46:05.312447 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:05.312407 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f4bb644f5-9cwg5_0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a/manager/0.log" Jul 14 18:46:05.779223 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:05.779145 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-ffbk7_07baa099-c4a9-4cec-8b71-fd04badfec54/copy-manifests/0.log" Jul 14 18:46:05.786874 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:05.786850 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-ffbk7_07baa099-c4a9-4cec-8b71-fd04badfec54/manager/0.log" Jul 14 18:46:06.289885 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:06.289858 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-67bf4f8764-8nhhp_631cbb70-13c3-4850-83a9-c056bf670e27/maas-api/0.log" Jul 14 18:46:06.621574 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:06.621545 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734245-9btn9_f964a77e-ea1b-44cc-89f0-114b855ca682/cleanup/0.log" Jul 14 18:46:06.743955 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:06.743924 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-296e74aa-hrf4t_801d0b7f-2ddf-46cb-a5cb-a520af544e19/revoke-keys/0.log" Jul 14 18:46:06.971179 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:06.971100 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-e0452a-jhqzx_b01f87f1-3cb2-4a9f-a2a3-a9889a0b706a/revoke-keys/0.log" Jul 14 18:46:07.083471 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:07.083444 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-b418a4-svfr4_3f43aa09-e9cb-4905-b5e6-9498194e738f/revoke-keys/0.log" Jul 14 18:46:07.313046 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:07.313017 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-4785d6-2pfdq_9f552588-312a-43ed-969f-30dc22ce74c8/revoke-keys/0.log" Jul 14 18:46:07.427657 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:07.427633 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-55e12d-r6x2w_e1dc631e-980a-421f-b646-1b2f2dec6bb8/revoke-keys/0.log" Jul 14 18:46:07.886101 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:07.886075 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-7e7ccb-ks7rl_66851c07-3714-4848-8b46-e2c986092b63/revoke-keys/0.log" Jul 14 18:46:08.111445 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:08.111412 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-dnrnq_f92f4bee-41a4-4add-8257-baa8ea9632ac/postgres/0.log" Jul 14 18:46:09.608228 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:09.608189 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-c92gl_8c0bc7c6-9e4e-4244-a856-1229980c8c73/manager/0.log" Jul 14 18:46:09.725599 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:09.725560 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2485h_78fc3549-d46c-4343-9bbd-d27f67c53fc6/manager/0.log" Jul 14 18:46:09.832845 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:09.832814 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8brd5_ea7ac2de-c1c5-47a7-baf8-d507ede55274/kuadrant-console-plugin/0.log" Jul 14 18:46:09.960477 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:09.960383 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xl6l2_4fb65699-86b8-4ed2-a549-dca404c03a49/registry-server/0.log" Jul 14 18:46:11.199455 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:11.199400 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/1.log" Jul 14 18:46:11.340112 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:11.340082 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-httf6_6108e278-8882-49da-9e5a-8e016c95b5a0/payload-pre-processing/0.log" Jul 14 18:46:11.460177 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:11.460076 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kvwjb_2c1ba809-dcf1-451c-b202-a01858d6d106/payload-processing/0.log" Jul 14 18:46:11.939624 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:11.939576 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tn6jq_06eda4d9-d49e-4b98-9ee8-53a660331709/main/0.log" Jul 14 18:46:12.184686 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:12.184643 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-x824q_5726ac0f-d7e7-4779-a12b-3d42346bd3f0/main/0.log" Jul 14 18:46:12.308065 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:12.308036 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh7h9z_caa430e5-ded9-49b6-b7d8-0e82d6e84e10/main/0.log" Jul 14 18:46:12.424191 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:12.424163 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rwrxv_ede9519b-a724-46d0-a9eb-75b8793c9799/main/0.log" Jul 14 18:46:16.536944 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:16.536907 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:46:16.637516 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:16.637462 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" limit=1024 Jul 14 18:46:21.032623 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:21.032578 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k2vhg_8b715d57-a5dc-4ab9-a089-52a1c8919e91/global-pull-secret-syncer/0.log" Jul 14 18:46:21.145335 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:21.145302 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mxgj8_c0496d62-71e1-4e18-a9b2-d07898e8460e/konnectivity-agent/0.log" Jul 14 18:46:21.326836 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:21.326805 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-115.ec2.internal_f9297309d9edb2e81185f4bef95a6103/haproxy/0.log" Jul 14 18:46:26.966996 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:26.966967 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-c92gl_8c0bc7c6-9e4e-4244-a856-1229980c8c73/manager/0.log" Jul 14 18:46:27.015364 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:27.015329 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2485h_78fc3549-d46c-4343-9bbd-d27f67c53fc6/manager/0.log" Jul 14 18:46:27.083809 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:27.083771 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8brd5_ea7ac2de-c1c5-47a7-baf8-d507ede55274/kuadrant-console-plugin/0.log" Jul 14 18:46:27.174032 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:27.174000 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xl6l2_4fb65699-86b8-4ed2-a549-dca404c03a49/registry-server/0.log" Jul 14 18:46:29.183669 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.183635 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/alertmanager/0.log" Jul 14 18:46:29.238197 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.238166 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/config-reloader/0.log" Jul 14 18:46:29.282786 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.282755 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/kube-rbac-proxy-web/0.log" Jul 14 18:46:29.330097 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.330064 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/kube-rbac-proxy/0.log" Jul 14 18:46:29.357898 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.357871 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/kube-rbac-proxy-metric/0.log" Jul 14 18:46:29.389537 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.389506 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/prom-label-proxy/0.log" Jul 14 18:46:29.421859 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.421819 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_995afcba-5533-4270-9dbe-f3fdde9dd0f8/init-config-reloader/0.log" Jul 14 18:46:29.492572 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.492463 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-dp9sr_57e2a584-e4f2-4c7b-ad1c-c19bed20c47f/cluster-monitoring-operator/0.log" Jul 14 18:46:29.528710 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.528681 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-xdzxs_7dbd9b71-d0de-48ed-9902-d02298af18be/kube-state-metrics/0.log" Jul 14 18:46:29.553206 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.553181 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-xdzxs_7dbd9b71-d0de-48ed-9902-d02298af18be/kube-rbac-proxy-main/0.log" Jul 14 18:46:29.590964 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.590934 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-xdzxs_7dbd9b71-d0de-48ed-9902-d02298af18be/kube-rbac-proxy-self/0.log" Jul 14 18:46:29.918834 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.918809 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s7tb7_62ac6b76-e411-4c43-8719-76e731d6ddd5/node-exporter/0.log" Jul 14 18:46:29.999749 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:29.999717 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s7tb7_62ac6b76-e411-4c43-8719-76e731d6ddd5/kube-rbac-proxy/0.log" Jul 14 18:46:30.045480 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.045453 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s7tb7_62ac6b76-e411-4c43-8719-76e731d6ddd5/init-textfile/0.log" Jul 14 18:46:30.079138 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.079110 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wtvlf_9b9dd58d-05c5-4636-8ada-2dfeb3569c9c/kube-rbac-proxy-main/0.log" Jul 14 18:46:30.107903 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.107870 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wtvlf_9b9dd58d-05c5-4636-8ada-2dfeb3569c9c/kube-rbac-proxy-self/0.log" Jul 14 18:46:30.130986 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.130952 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wtvlf_9b9dd58d-05c5-4636-8ada-2dfeb3569c9c/openshift-state-metrics/0.log" Jul 14 18:46:30.380505 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.380474 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-hxmn6_5f1dac4a-ae12-46d7-a951-3d89dd37f481/prometheus-operator/0.log" Jul 14 18:46:30.404681 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.404649 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-hxmn6_5f1dac4a-ae12-46d7-a951-3d89dd37f481/kube-rbac-proxy/0.log" Jul 14 18:46:30.474801 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.474765 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66897b9758-7blqq_58c00766-dc64-4549-83d3-e3364ef1e6b6/telemeter-client/0.log" Jul 14 18:46:30.500173 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.500148 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66897b9758-7blqq_58c00766-dc64-4549-83d3-e3364ef1e6b6/reload/0.log" Jul 14 18:46:30.525435 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:30.525406 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66897b9758-7blqq_58c00766-dc64-4549-83d3-e3364ef1e6b6/kube-rbac-proxy/0.log" Jul 14 18:46:31.746469 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.746433 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x"] Jul 14 18:46:31.746887 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.746873 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66851c07-3714-4848-8b46-e2c986092b63" containerName="revoke-keys" Jul 14 18:46:31.746931 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.746889 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="66851c07-3714-4848-8b46-e2c986092b63" containerName="revoke-keys" Jul 14 18:46:31.746965 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.746958 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="66851c07-3714-4848-8b46-e2c986092b63" containerName="revoke-keys" Jul 14 18:46:31.750377 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.750356 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.752939 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.752918 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qkbmz\"/\"kube-root-ca.crt\"" Jul 14 18:46:31.753046 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.752918 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qkbmz\"/\"openshift-service-ca.crt\"" Jul 14 18:46:31.753663 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.753646 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qkbmz\"/\"default-dockercfg-qfjll\"" Jul 14 18:46:31.758800 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.758778 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x"] Jul 14 18:46:31.779784 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.779752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-proc\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.779957 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.779798 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-lib-modules\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.779957 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.779827 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-sys\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.779957 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.779922 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-podres\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.780142 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.780015 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgj9f\" (UniqueName: \"kubernetes.io/projected/571a1f65-5be6-45e7-b975-a9e2c4b12d88-kube-api-access-jgj9f\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.880817 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880777 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-proc\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.880817 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-lib-modules\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880836 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-sys\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-podres\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880905 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgj9f\" (UniqueName: \"kubernetes.io/projected/571a1f65-5be6-45e7-b975-a9e2c4b12d88-kube-api-access-jgj9f\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880915 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-proc\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880923 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-sys\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.880990 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-lib-modules\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.881070 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.881049 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/571a1f65-5be6-45e7-b975-a9e2c4b12d88-podres\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:31.890713 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:31.890687 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgj9f\" (UniqueName: \"kubernetes.io/projected/571a1f65-5be6-45e7-b975-a9e2c4b12d88-kube-api-access-jgj9f\") pod \"perf-node-gather-daemonset-kt56x\" (UID: \"571a1f65-5be6-45e7-b975-a9e2c4b12d88\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:32.061575 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:32.061544 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:32.206776 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:32.206742 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x"] Jul 14 18:46:32.209704 ip-10-0-129-115 kubenswrapper[2572]: W0714 18:46:32.209676 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod571a1f65_5be6_45e7_b975_a9e2c4b12d88.slice/crio-6a68e396bfdcb6d1fec9472157dbe1d79a66ee344e11bfea31270bf3043471b6 WatchSource:0}: Error finding container 6a68e396bfdcb6d1fec9472157dbe1d79a66ee344e11bfea31270bf3043471b6: Status 404 returned error can't find the container with id 6a68e396bfdcb6d1fec9472157dbe1d79a66ee344e11bfea31270bf3043471b6 Jul 14 18:46:33.020553 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:33.020516 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" event={"ID":"571a1f65-5be6-45e7-b975-a9e2c4b12d88","Type":"ContainerStarted","Data":"8571fa9e39d508b73085d881887eea9daf7c07995fb18041b99f6da4388a7f45"} Jul 14 18:46:33.020949 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:33.020571 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" event={"ID":"571a1f65-5be6-45e7-b975-a9e2c4b12d88","Type":"ContainerStarted","Data":"6a68e396bfdcb6d1fec9472157dbe1d79a66ee344e11bfea31270bf3043471b6"} Jul 14 18:46:33.020949 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:33.020592 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:35.108241 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:35.108210 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x674q_c3e8c4ef-5559-4a7d-936b-8122e30e432b/dns/0.log" Jul 14 18:46:35.137969 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:35.137933 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x674q_c3e8c4ef-5559-4a7d-936b-8122e30e432b/kube-rbac-proxy/0.log" Jul 14 18:46:35.209878 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:35.209844 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5q98w_03a7cf65-9266-4371-8921-8568aeffe9d2/dns-node-resolver/0.log" Jul 14 18:46:36.076356 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:36.076317 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-brnmt_b55e97e6-bdd1-435b-8a18-1975c8640be7/node-ca/0.log" Jul 14 18:46:37.602874 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:37.602842 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/0.log" Jul 14 18:46:37.657385 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:37.657353 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/1.log" Jul 14 18:46:37.763599 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:37.763560 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-httf6_6108e278-8882-49da-9e5a-8e016c95b5a0/payload-pre-processing/0.log" Jul 14 18:46:37.822354 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:37.822320 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kvwjb_2c1ba809-dcf1-451c-b202-a01858d6d106/payload-processing/0.log" Jul 14 18:46:38.751547 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:38.751461 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k5bwr_478be86d-41ba-406e-a434-cbc62ac47c65/serve-healthcheck-canary/0.log" Jul 14 18:46:39.036075 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.035998 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" Jul 14 18:46:39.064604 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.064552 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-kt56x" podStartSLOduration=8.064537334 podStartE2EDuration="8.064537334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 18:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:46:33.049053625 +0000 UTC m=+3135.911843440" watchObservedRunningTime="2026-07-14 18:46:39.064537334 +0000 UTC m=+3141.927327137" Jul 14 18:46:39.359449 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.359415 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-jgsr8_81c45dce-e688-45f9-b692-672783e31d68/insights-operator/0.log" Jul 14 18:46:39.360718 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.360697 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-jgsr8_81c45dce-e688-45f9-b692-672783e31d68/insights-operator/1.log" Jul 14 18:46:39.483097 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.483065 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njpf6_015e5b9a-80c0-472a-b3f7-86de6814895c/kube-rbac-proxy/0.log" Jul 14 18:46:39.513580 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.513551 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njpf6_015e5b9a-80c0-472a-b3f7-86de6814895c/exporter/0.log" Jul 14 18:46:39.547981 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:39.547944 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njpf6_015e5b9a-80c0-472a-b3f7-86de6814895c/extractor/0.log" Jul 14 18:46:41.885536 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:41.885505 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/0.log" Jul 14 18:46:41.941254 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:41.941221 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/1.log" Jul 14 18:46:41.973675 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:41.973643 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-httf6_6108e278-8882-49da-9e5a-8e016c95b5a0/payload-pre-processing/0.log" Jul 14 18:46:42.013896 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:42.013865 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kvwjb_2c1ba809-dcf1-451c-b202-a01858d6d106/payload-processing/0.log" Jul 14 18:46:43.033345 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.033305 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f4bb644f5-9cwg5_0cf5a7b4-304d-4d33-aa11-c0a8cf94af9a/manager/0.log" Jul 14 18:46:43.377727 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.377696 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-ffbk7_07baa099-c4a9-4cec-8b71-fd04badfec54/manager/0.log" Jul 14 18:46:43.440022 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.439994 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-ffbk7_07baa099-c4a9-4cec-8b71-fd04badfec54/copy-manifests/0.log" Jul 14 18:46:43.609394 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.609362 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/0.log" Jul 14 18:46:43.665811 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.665730 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6t6m6_ed28cebc-3944-46b3-92bf-946678bc3030/istio-proxy/1.log" Jul 14 18:46:43.709954 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.709923 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-httf6_6108e278-8882-49da-9e5a-8e016c95b5a0/payload-pre-processing/0.log" Jul 14 18:46:43.747132 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:43.747108 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kvwjb_2c1ba809-dcf1-451c-b202-a01858d6d106/payload-processing/0.log" Jul 14 18:46:44.895333 ip-10-0-129-115 kubenswrapper[2572]: I0714 18:46:44.895302 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57cf6fdf7d-kcws4_3984a61e-a1ec-4916-ac8a-c2afc15376da/manager/0.log"